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ABSTRACT 

Guidelines  are  presented  for  Achieving  human  factors 
Inputs  to  the  design  of  synthetic  training  systems.  A method  is 
developed  for  design  and  organization  of  training  concepts  and  data 
supportive  to  the  human  factors  specialist  in  deriving  the  functional 
specifications  for  the  design  of  any  complex  training  device.  Three 
major  sections  are  provided.  The  first  of  these  presents  an  organized 
method  for  achieving  human  factors  inputs  to  training  system  design. 
Another  section  presents  concepts  and  data  applicable  to  the  design 
of  training  devices — visual  simulation,  platform  simulation,  vehicle 
control  requirements,  information  processing  requirements, 
measurement  system  design,  adaptive  training  strategies,  and 
deliberate  departure  from  realism  in  design.  Design  support  for  each 
of  these  factors  is  articulated  based  on  a review  of  the  pertinent 
literature,  where  design  evidence  is  meager,  the  data  gaps  are 
identified.  Research  issues  of  high  priority  for  human  factors  design 
are  recommended.  The  third  section  contains  a demonstration  of  the 
human  factors  design  process  for  a complex  training  system. 
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FOREWORD 


Over  the  past  quarter  century,  the  literature  on  training  devices  has 
grown  to  voluminous  proportions.  A glance  at  the  bibliography  in  this 
report  will  confirm  this.  It  was  the  purpose  of  this  project  to  select, 
out  of  this  welter  of  information,  what  was  sound,  what  was  based  on  prin- 
ciples of  human  learning,  what  was  supported  by  hard  data,  and  to  put  this 
together  within  two  covers  to  provide  a source  book  and  reference  guide  to 
the  design  and  construction  of  training  devices.  This  volume  is  designed 
to  serve  the  needs  of  all  those  concerned  with  the  development  of  training 
devices.  Project  Officers  and  Engineers  and  Human  Factors  Personnel  who 
round  out  the  device  team. 

Dr.  Alfred  F.  Smode  of  Dunlap  and  Associates  gathered  the  information, 
organized  it  in  its  present  form,  and  wrote  the  report.  Dr.  Knox  E.  Miller 
of  the  NAVTRADEVCEN  devised  the  method  for  developing  visual  simulation 
specifications  and  described  it  in  Section  3.1.3.  Dr.  Hugh  M.  Bowen  of 
Dunlap  and  Associates  wrote  the  rest  of  the  Chapter  on  Visual  Simulation, 
and  Dr.  Herbert  Smithline  of  Dunlap  and  Associates  contributed  the  Chapter 
on  Vehicle  Control  Requirements. 


VINCENT  J.  SHARKEY 
Psychologist 
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SECTION  I 


I.  INTRODUCTION 


IN  T RODUC  TION 


1.  I PURPOSE.  This  report  presents  guidelines  for  human  factors  inputs 
to  the  design  of  synthetic  training  systems  . It  provides  a method  for  design^- 
and  organizes  training  concepts  and  data  supportive  to  the  human  factors 
specialist  in  achieving  the  functional  specifications  for  the  design  of  any 
complex  synthetic  training  system  wherein  the  training  device  ( simulator) 
is  the  focal  point  of  the  instructional  system,  i.e.  , the  primary  means  for 
training . 


The  method  is  constructed  to  facilitate  the  rapid  development  of  a 
human  factors  design  pathway  by  making  explicit  the  data  application  points 
and  the  alternatives  available  for  achieving  the  design  concept  for  any  train- 
ing system  under  consideration.  In  essence,  the  method  is  devoted  to  pro- 
cedures and  techniques  that  may  be  used  to  advantage  in  training  device 
design;  guideline  information  (concepts  and  data)  is  also  provided  to  assist 
the  human  factors  specialist  in  identifying  those  design  strategies  and  options 
which  will  enhance  the  transfer  of  training  achieved  via  simulation. 


1.2  PERSPECTIVE.  The  emphasis  in  this  presentation  is  on  human 
factors  inputs  as  elements  in  the  training  device  design  process  and  is  con- 
cerned with  the  design  and  utilization  of  the  simulator  as  a training  tool. 

It  centers  on  the  issues  of  human  factors  responsibility  in  device  design, 
and  on  the  information  and  data  needed  to  fulfill  this  responsibility,  speci- 
fically, what  information  is  immediately  available  for  design  use,  how  well 
the  findings  of  research  translate  to  the  specific  problems  of  design,  what 
data  gaps  hamper  efficient  design  decisions;  and  the  methods  ( procedure s , 
techniques)  available  to  enable  the  rapid  development  of  a design  pathway 
for  specifying  a relevant  and  effective  training  system. 

The  human  factors  goal  is  to  provide  an  instructional  capability  con- 
sistent with  the  defined  purpose  of  the  training  device.  The  requirement 
is  easily  described:  provide  a design  to  maximize  transfer  of  training 

(correlation  of  performances  in  the  simulator  with  the  performances  re- 
quired in  the  real  world  counterpart).  This  involves  the  capability  for 
structuring  training  (hardware  and  software)  so  that  critical  system/ mission 


This  method  is  applicable  generally  to  any  training  device  design  effort; 
it  in  also  compatible  with  the  Kaval  Training  Device  Center’s  currently 
used  Military  Characteristics  (MC)  Instruction  (NAVTRADEVGEN 
INSTRUCTION  3916.4,  : July  1969)  • The  purpose  of  an  MC  Document  is 
to  identify  a training  problem,  and,  when  required,  to  define  and  describe 
the  functional  training  characteristics,  of  a device  required  to  provide  the 
needed  training  capability. 
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events  can  be  installed  and  controlled  to  occur  in  prescribed  ways  at  pre- 
scribed times.  Flexibility  in  task  representation  and  control  with  the 
capability  to  monitor , evaluate  and  score  performance  is  the  basis  for  the 
development  of  training  strategies.  The  design  objective  is  to  specify  the 
characteristics  of  the  pertinent  operational  environment  to  simulate  for 
training,  and  the  human  factors  effort  concentrates  on  selecting  the  design 
alternative(s)  which  will  define  the  manner  in  which  the  synthetic  environ- 
ment for  training  will  be  provided  and  how  learning  will  be  shaped  (struc- 
tured and  controlled).  This  embraces  two  distinct  but  interrelated  opera- 
tions: 1)  trainee  station  design,  which  considers  most  prominently  the 

issues  involved  in  achieving  fidelity  of  simulation,  that  is,  specifying  the 
characteristics  of  hardware  (display s / controls)  and  the  functions  with  which 
the  trainee  must  interact  in  the  device  and  their  fidelity  of  representation; 
and  2)  the  management  of  training,  which  concerns  the  structuring,  con- 
trolling, and  monitoring  of  training  at  the  instructor  station.  Achievement 
of  these  design  requirements  is  the  subject  of  this  document* 


Material  is  presented  to  provide  design  guidance  pertinent  to  most 
classes  of  training  devices  with  the  emphasis  on  complex  weapons  system 
trainers.  By  concentrating  on  the  more  difficult  classes  of  training  devices 
it  is  our  belief  that  the  requirements  for  more  simple  {training  devices  and 
part-task  trailers  can  be  handled  equally  well  via  the  content  of  this  docu- 
ment (i.e.  , design  options  for  more  difficult  or  complex  problems  serve  as 
models  for  solving  less  stringent  problems).  Thus,  the  guidelines  are 
applicable  to  the  major  classes  of  training  devices  currently  in  the  Navy 
inventory . 


The  document  is  directed  primarily  towards  individuals  charged  with 
the  responsibility  for  human  factors  inputs  during  the  conception  and  devel- 
opment of  the  design  of  training  devices.  It  is  our  assumption  that  in  the 
design  of  a specific  training  device  this  document  will  be  of  maximum  value 
to  the  human  factor  s , specialist  who,  1)  is  already  skillful  in  the  design 
process,  and  2)  is  or  will  become  intimately  familiar  with  the  nature  of, 
and  the  characteristics  of  the  specific  system  under  consideration.  This 
is  in  keeping  with  our  intent  to  provide  guidance  and  support  (i.e.  , guide- 
book, not  handbook,  format)  to  the  human  factors  effort  in  any  device  desiga 


1.3  PROBLEMS  IN  GUIDELINEwS  DEVELOPMENT . As  a prelude  to  the 
presentation  of  the  material  in  this  guide , we  recoghize  the  substantial 
hazards  associated  with  organizing  guidelines  for  human  factors  inputs  to 
training  device  design.  Any  attempt  to  formalize  the  strategies  and  the 
available  design  options  to  optimize  the  instructional  value  of  training 
devices  and  to  provide  guidelines  of  general  utility  which  will  satisfy  the 
requirement  of  ‘’something  for  any  situation* 1 is  frought  with  difficulties, 
both  in  content  and  format.  One  class  of  difficulty  is  the  lack  of  standard 


procedures  in  device  development.  Each  procurement  is  handled  accord- 
ing to  its  own  merits  and  constraints.  Foremost  among  these  is  the  type 
0 perational  requirement  / system  to  which  the  device  pertains,  the 
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purpose  to  be  achieved  by  the  device,  and  the  design  information  already 
available  (e.g.  , an  operational  system  vs.  a system  in  some  stage  of 
development).  Another  compelling  difficulty  is  that  today  no  method  pre- 
vails which  demonstrates  convincingly  a means  for  exploiting  information 
that  will  yield  consistently  effective  solutions  to  training  problems. 

Thus,  our  development  of  the  issues  and  problems  which  influence 
the  selection  of  human  factors  design  options  has  been  constrained  by  the 
following  features. 

a.  The  methods  for  achieving  human  factors  design  solutions 
are  inexact.  We  have  attempted  to  assemble  the  most 
coherent  approaches  to  design.  These  unfortunately  are 
of  differential  usefulness,  therefore,  the  recommendations 
are  of  differential  utility. 

b.  The  information  base  of  concepts  and  data  pertinent  to  design 
is  weak.  We  have  attempted  to  select  the  research  results 
most  applicable  to  simulator  design  (for  example,  threshold 
data  on  motion  perception  rather  than  data  on  tolerances  to 
excessive  G) . Also,  researches  yielding  generalizable  data 
were  sought.  Excluded  were  studies  requiring  major  ex- 
trapolations to  bring  the  findings  in  line  with  design  utility. 

c.  Considerable  gaps  in  design  data  and  information  of  marginal 
value  exist.  We  have  attempted  to  describe  the  weaknesses 
in  the  data  base  and  have  recommended  research  issues  of 
foreseeable  priority  based  on  the  status  of  research  in  defined 
areas . 

1.4  THE  ISSUE  OF  FIDELITY.  Ingenuity  is  required  in  assembling  a 
synthetic  representation  of  some  portion  of  the  real  world  in  a way  that  will 
train  a man  to  perform  better  in  that  real  world.  The  view  of  the  simulator 
as  a training  tool  places  a premium  on  transfer  of  training  (via  the  selec- 
tion of  design  alternatives  which  consider  both  cost  and  training  technology). 
Historically,  simulation  design  has  been  preoccupied  with  the  problems  of 
achieving  maximum  fidelity  of  simulation.  This  quest  for  physical  corres- 
pondence to  the  operational  environment  (i.e.  , engineering  fidelity  in 
representation)  has  often  placed  the  development  of  a relevant  training  pro- 
gram and  instructional  strategies  in  a secondary  design  role.  In  fact, 
some  of  the  design  practices  in  the  attempts  to  achieve  this  "real  world11 
physical  equivalence  have  at  times  increased  costs  with  no  corresponding 
increase  in  training  value  and  in  some  instances  have  actually  interfered 
with  realizing  the  full  training  potential  of  simulators. 

A question  of  importance  in  this  report  concerns  the  role  of  the  human 
factors  specialist  in  defining  the  requirements  for  fidelity  of  simulation  in 
. raining  system.  Beginning  with  the  assumption  that  costs  and  the 
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engineering  state-of-the-art  are  the  overriding  constraints  in  providing  the 
environment  in  which  learning  can  be  made  to  occur,  how  is  fidelity  most 
effectively  correlated  with  instructional  strategy?  The  human  factors 
specialist  must  consider  fidelity  of  simulation  in  terms  of  structure  and 
control  of  training  (based  on  the  training  requirements  analysis  as  described 
in  Section  II  of  this  report)*  In  structuring  the  environment  for  learning, 
the  fidelity  of  representation  must  fulfill  the  training  and  utilization  expec- 
tancies for  the  device.  Maximum  engineering  fidelity  is  specified  to  the 
extent  that  it  facilitates  the  development  of  desired  performances,  however, 
well  designed  simulators  also  deviate  intentionally  from  operational  realism 
in  order  to  promote  learning.  Thus,  several  approaches  to  fidelity  must  be 
considered  interactively  in  design  and  played  against  the  purpose  and  objec- 
tives of  the  contemplated  training  (see  Section  2.7.2): 


a.  The  Extent  of  Simulation  Required — this  refers  to  the  what  (amount) 
of  simulation  desired  ( i . e . , the  inclusion  or  exclusion  of  a simulation  ele- 
ment). The  issue  concerns  what  should  be  simulated  (attainable  in  state-of- 
the-art  engineering  terms)  to  provide  an  environment  necessary  to  achieve 
the  defined  purpose  and  objectives  of  training.  An  example  of  such  a design 
alternative  is  the  option  to  provide  knuckles  and  wakes  in  sonar  simulation 
as  sources  of  false  targets. 


b.  The  Degree  of  Fidelity  Required — this  refers  to  the  degree  of 
physical  correspondence  to  the  operational  environment  and  concerns  the 
fidelity  levels  required  to  achieve  effective  training.  Involved  is  a continuum 
of  engineering  fidelity  ranging  from  a selection  of  alternatives  when  the 
state-of-the-art  is  adequate  to  achieve  any  of  a number  of  definable  options, 
to  a reduction  in  fidelity  tolerances  when  the  engineering  state-of-the-art 

is  less  than  adequate.  The  most  efficient  selection  of  degree  of  fidelity 
should  be  based  on  man’s  perceptual  requirements,  i.e.  , perceptual  equiv- 
alence to  the  operational  environment.  All  other  things  equal,  reduced 
engineering  fidelity  with  attendant  cost  reductions  is  desired  when  it  can  be 
demonstrated  that  training  effectiveness  is  not  compromised. 

c.  Deliberate  Departure  From  Realism — this  refers  to  intentional 
gross  deviations  in  configuration/ ope  ration  from  that  found  in  the  opera- 
tional system  or  environment  being  simulated.  Representative  techniques 
include  the  use  of  signal  enhancement,  information  feedback  about  perform- 
ance, and  cues  and  prompts  in  equipment  design. 


The  human  factors  specialist  's  less  concerned  with  the  method  of 
simulation  than  with  what  is  represented  at  the  display  and  control  inter- 
faces. His  interest  in  fidelity  of  simulation  is  in  terms  of  the  trainee  *s 
interaction  with  the  instructional  system  and  in  the  training  process  for 
maximizing  transfer  of  training . He  must  know  the  operational  environ- 
ment in  order  tomake  judgments  about  needed  simulation  elenaents  to  pro- 
vide the  training  capability;  knowledge  of  the  operational  environment  is 
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necessary  to  define  alternative  ways  of  representing  the  environment  in 
which,  training  will  be  accomplished.  He  must  also  know  the  trainee  1 s 
capabilities,  vis-a-vis,  the  design  requirements  (for  example , the  decision 
on  the  number  of  targets  to  present  is  most  meaningful  in  terms  of  how 
much  information  the  trainee  can  handle  and  process). 

1.4.  1 Computer  Advances.  With  increasingly  abundant  computer  capacity, 
it  appears  quite  useful  not  to  "skimp1*  on  high  fidelity  as  an  approach  to 
design  (although  this  runs  counter  to  a minimum  initial  cost  design  philos- 
ophy). At  issue  is  the  provision  of  maximum  flexibility  in  trainer  usage. 

For  example,  the  inclusion  of  more  than  the  minimum  in  simulation  ele- 
ments may  heighten  considerably  the  training  capability  and  flexibility  of 
the  simulator  (heightened  engineering  fidelity  and  complete  representation 
may  be  extremely  desirable  in  a number  of  subsystem  areas).  Careful 
consideration  of  increased  capability  and  flexibility  in  design,  with  the 
attendant  costs,  may  minimize  later  requirements  for  expensive  modifica- 
tions to  a training  device  once  it  is  on-line  and  training  experienc e (hind- 
sight) is  obtained  with  it. 


The  advances  in  computer,  capabilities  have  solved  some  of  the 
design  problems  of  an  earlier  time  (e.g.  , target  characteristics,  maneuv- 
erability, number  of  targets  displayed,  etc.)  and  excess  computer  capacity 
can  be  used  to  minimize  existing  design  problems  . Hunt  (1967)  suggests 
that  the  substitution  of  an  increasing  computer  capacity  for  expensive  en- 
gineering ingenuity  is  inevitable.  He  suggests,  for  example,  that  the  gen- 
eration of  an  acceptable  mathematical  model  for  aerodynamic  and  engine 
performance  can  be  considerably  simplified  by  usage  of  large  numbers  of 
multi-variable  functions  (requiring  considerable  computer  capacity). 

Lavish  computational  power  (e.  g.  , extremely  rapid  computation)  can  be 
utilized  to  resolve  some  prominent  and  long  standing  problems,  in  visual 
and  in  motion  simulation,  in  realistic  simulation  of  control  forces  through- 
out all  operating  conditions  , in  radar  land  mass  simulation  (i.e.  , digital 
storage  of  the  massive  data  file)  , and  in  realistic  generation  of  ECM  ex- 
pendables (e.  g.,  chaff  bundles)  . 


A basic  issue  is  to  determine  the  most  desirable  fidelity  levels  for 
the  simulation  parameters  when  the  engineering  state-of-the-art  is  ade- 
quate, and  the  fidelity  levels  that  are  achievable  when  reduction  in  tolerances 
is  required  because  of  inadequacies  in  the  engineering  state-of-the-art. 

These  deliberations  must  consider  the  following  in  relation  to  the  real 
world:  1)  idealized  representation,  for  example,  the  best  equations  ob- 

tainable.; 2)  realistic  representation,  for  example , simplified  equations 
involving  backing-off  from  fidelity  for  the  purpose  of  economy,  and  3)  im- 
plementation of  the  selected  fidelity  option,  i.  e . , an  additional  attenuation 
in  fidelity  resultingvfrom  mechanization. 


1.5  ORGANIZATION  OF  THE  REPORT.  In  order  to  arrange  and  discuss 
O wide  content  range  selected  for  this  guid^e , three  major  sections  are 
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presented,  in  addition  to  the  Introductory  Section.  Section  II  provides  a 
method  for  achieving  human  factors  inputs  to  training  system  design. 

Section  III  presents  concepts  and  data  applicable  to  the  design  of  training 
systems  . Section  IV  presents  a demonstration  of  human  factors  recom- 
mendations in  the  design  of  a synthetic  training  system.  This  is  a recon- 
struction of  the  human  factors  design  process  in  achieving  device  specifi- 
cations (for  device  14A2,  ASROC/ASW  Early  Attack  Weapon  System 
Trainer),  and  is  based  on  the  method  depicted  in  Section  II. 

Seven  content  chapters  are  subsumed  under  Section  III.  These  are: 

Visual  simulation 
Platform  motion  simulation 
Vehicle  control  requirements 
Information  processing  requirements 
Measurement  system  design 
Adaptive  training  strategies 

Deliberate  departures  from  realism  in  design 

For  each  chapter,  concepts  and  data  which  support  the  human  factors  effort 
in  design  are  articulated  based  on  a review  of  the  pertinent  literature. 

Where  design  evidence  is  meager,  the  data  gaps  are  identified.  Finally, 
research  issues  that  are  of  priority  for  human  factors  design  of  training 
devices  are  recommended.  The  philosophy  underlying  the  research  recom- 
mendations is  that  each  issue  outlined  is  either  1)  specific  to  an  obvious 
need  in  device  design,  or  2)  intended  to  increase  the  understanding  of  human 
behavior  which  will  prove  fruitful  in  s olving  training  problems  not  easily 
examined. 
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SECTION  II 


METHOD  FOR  ACHIEVING  HUMAN  FACTORS 
INPUTS  TO  TRAINING  SYSTEM  DESIGN 

2.  INTRODUCTION 


The  idealized  training  system  development  process  is  made  up  of  a 
number  of  phases  which,  describe  a chronological  interrelated  series  of 
steps,  each  involving  specific  human  factors  considerations.  The  goal  is 
straightforward:  to  build  a synthetic  system  that  provides  training  in  the 

most  efficient  manner  relative  to  the  fleet’s  needs.  Unfortunately,  the 
means  for  achieving  this  goal  are  not  straightforward.  There  are  gaps  in 
training  technology  relative  to  the  design  of  synthetic  devices;  there  are 
also  considerations  of  tradeoffs  between  costs  and  training  effectiveness 
that  yield  functional  design  based  on  sources  other  than  the  job  requirements. 


Within  these  constraints,  a number  of  human  factors  techniques  have 
evolved  that  have  been  applied  to  the  design  of  the  various  classes  of  complex 
synthetic  training  systems  on-line  today.  Overall,  a style  has  emerged 
which  utilizes  a systems  approach  to  the  design  of  training  systems.  Broad- 
ly, this  approach  involves  a sequence  of  steps  in  the  training  device  design 
process  beginning  with  the  initial  determination  of  the  requirements  for 
training,  proceding  through  hardware  and  software  design  (in  terms  of 
fidelity  of  simulation  requirements  in  the  trainee  station(s)  and  in  the  in- 
structor station  design)  , and  ideally,  ending  with  the  development  of  utili- 
zation procedures  to  insure  effective  training  within  the  capabilities  of 
device  design. 


Unfortunately,  this  basic  methodology  for  human  factors  design  of  train- 
ing systems  is  sketchy,  incomplete,  or  inexact  and  has  not  met  the  expecta- 
tions of  the  people  responsible  for  the  development  and  design  of  devices, 
or  for  the  development  of  utilization  procedures  once  the  device  is  delivered 
and  on-line.  The  best  of  the  methods  , those  that  have  proved  most  effective 
in  achieving  good  design  properties  for  training,  are  cumbersome  and  time 
consuming;  quite  often  the  outputs  of  the  human  factors  effort  are  too  late  to 
substantially  influence  certain  training  features  in  device  design. 


The  methods  available  today  are  not  complete  so  far  as  satisfying  all 
human  factors  requirements  in  the  training  device  design  process  . At  best, 
a given  method  may  emphasize  certain  important  aspects  of  the  human 
factors  design  process  but  be:  unable  to  effectively  account  for  other  equally 
important  aspects  . The  methods  available  also  demand  considerable  skill 
from  the  training  analyst  since  the  level  of  analysis  and  the  kinds  of  informa- 
tion required  for  a specific  training  system  are  only  generally  outlined; 
explicit  procedures  are  not  available  for  translating  task  description  data 
into  hardware  configuration  requirements. 
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A method  is  supplied  here  that  attempts  to  minimize  a number  of  the 
current  insufficiencies;  in  the  design  proces  s * It  organizes  much  of  the 
existing  useful  techniques  and  data  to  provide  the  human  factors  specialist 
support  in  the  development  of  the  training  requirements  information  to  the 
level  of  detail  needed,  consistent  with  the  development  of  the  functional 
specifications  for  the  training  system.  It  employs  a systems  approach^  to 
training  device  design  and  delineates  a series  of  chronological  phases  lead- 
ing to  design  specifications  for  optimizing  training.  This  method  is  based 
on  the  available  technology  of  training  and  assembles  workable  existing 
techniques  and  newer  concepts  into  a unified  human  factors  method  for 
achieving  training  system  design  specifications. 


The  method  is  not  inviolate . . Some  stage  s of  the  analyses  have  less 
clearly  defined  procedures  than  other  stages  because  of  weaknesses  in  the 
current  state-of-the-art  in  training  technology.  These  are  identified,.  The 
method  is  also  cumbersome  in  parts  and  requires  a qualified  training  anal- 
yst to  apply  the  procedures  . The  method  utilizes  the  available  sources  of 
information  such  as  system,  documentation,  doctrinal  publications,  inter- 
actions with  experts  in  the  operational  world  and  direct  observation.  ^ ; 


While  the  method  is  not  simple,  it  nevertheless  provides  the  necessary 
guidance  to  the  human  factors  specialist  to  do  his  job  in  the  most  efficient 
manner.  ^ What  is  set  up  is  a pathway  for  rapidly  coming  to  grips,  with  the 
essential  design  issues  pertinent  to  most  classes  of  major  training  systems 
The  emphasis  on  class  is  deliberate  since  there  is  such  variation  among 
training  devices  that  any  method  cannot  be  common  to  all,  There  are,  how- 
ever , commonalities  between  and  within  classes  of  training  systems  (.e.g.  , 
operational  flight  trainers,  tactical  team  trainers,  individualized  general 
trainers  , etc  . ) and  these  will  be  highlighted.  During  the  course  of  this 
development,  many  and  diverse  examples  will  be  shown  that  will  enable  the 
reader  to  generalize  to  a specific  system  under  development.  In  addition, 
a demonstration  of  the  method  applied  to  a current  on-line  training  device 
is  supplied  in  Section  IV  of  this  report  which  presents  a reconstruction  of 
the  human  factors  design  inputs  for  a complex  tactical  team  trainer. 


•^The  behaviors  which  must  be  exhibited  on  the  job  is  the  goal  of  the  training 
system.  To  achieve  this,  an  integrated  series  of  learning  experiences 
must  be  provided,  employing  an  organized  hardware/ software  configuratio 
to  produce  the  behaviors  required.  Measurement  is  provided  to  assure 
that  training  design  is  compatible  with  job  requirements  . The  process 
begins  with  defining  the  purposes  and  objectives  of  training;  these  provide 
a basis  for  deriving  training  content ; methods > measures  , and  training 
hardware  . The  data  are  also  useful  in  the  development  of  utilization 
procedures  for  the  trainer  ; ■ 

^The  method  yields  its  best  results  when  an  interdisciplinary  mix  of  human 
factors,  engineering,  and  operational  requirements  specialists  are  in- 
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The  major  phases  of  activity  which,  will  enable  a complete  and  adequate 
human  factors  contribution  to  the  functional  specifications  for:  a training 
device  are  outlined  below.  A pathway  is  set  up  for  rapidly  coming  to  grips 
with  the  essential  issues  in  functional  design  pertinent  to  any  class  of  pro  - 
posed  training  device.  The  varieties  of  issues  are  put  into  perspective 
within  a chronology  of  events  represented  in  the  following  phases  . 


a.  Definition  of  the  purpose  of  the  training  system. 

b.  Analysis  of  the  operational  system. 

c.  Analysis  of  tasks  involving  the  trainee(s). 


task  structure 


training  objectives 


'•  identification  of  simulation  elements  relative  to  own-vehicle 
characteristics,  target  characteristics,  and  the  media. 

d.  Gross  device  hardware  definition. 


e.  Definition  of  the  characteristics  of  the  operational  environment 
to  simulate  for  training. 

f.  Representing  the  operational  environment  in  the  trainee  compartment(s) 

g.  Provisions  for  the  management  of  training  at  the  instructor  station 
(design  for  the  structure  and  control  of  training)  . 


It  is  emphasized  that  the  pr  ocedures  for  obtaining  , , describing,  organizing 
and  correlating  information  for  device  design  are  iterative*  beginning  with 
gross  decisions  on  device;  requirements  and  culhainating  in  specifications  of 
the  functional  characteristics  for  the  device  to  provide  the  heed@d.  training 
capability.  The  series  of  activities  undertaken  begins  with  the  initial  de- 
velopment of  ideas  on  what  the  training  device  is  to  do  and  how  it  should  be 
constituted.  Prom  this  beginning,  a continual  refinement  and  explication  of 
the  ideas  is  accomplished. 

The  outputs  resulting  from  the  above  sequence  of  .events  are  detailed  next 
in  a series  of  five  sequential  and  iterative  phases  ^ ■ 

2. 1 PHASE  I:-  . DEFINITION  OF  THE ; PURPOSE  OF  THE  TRAINING 
DEVICE.  In  determining  the  need  for  a synthetic  training  system  to  meet 
fleet  requir  ements , an  initial  effort  is  centered  on  defining  clearly  the  pur  - 
pose of  training^  Articulate  statements  of  training  needs  provide  an  indica- 
tion of  the  magnitude  of  the  training  required  and  of  the  fidelity  level  of  the 
device  hardware  and  also  provide  a basis  for  what  to  simulate.  Members 
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of  the  design  team  must  interact  substantially  with  personnel  of  the  using 
facility  responsible  for  the  training  to  define  the  purpose  and  mission  of 
the  training. 

The  requirement  is  this:  define  the  design  pathway  and  achieve  initial 

(gross)  decisions  on  the  magnitude  of  training  required  and  on  the  fidelity 
level  desired  in  the  device  hardware,  and  also  provide  a basis  for  what  to 
simulate.  The  scope  of  the  training  is  outlined  and  that  "piece1 1 of  the 
operational  universe  to  be  represented  is  documented.  In  achieving  this, 
it  is  desirable  to  view  the  operational  equipment  relevant  to  the  training 
device;  if  the  system  is  not  yet  operational,  then  the  available  system  docu- 
mentation must  be  reviewed. 

2.1.1  Defining  the  Design  Pathway.  Initial  definition  of  the  design  path- 
way is  accomplished  by  means  of  decisions  made  within  the  following  gross 
categories. 

2. 1.  1.  1 Characteristics  of  the  Environment  for  Training  and  the  Trainee 
Involvement. 

a.  Type  of  training  device  (fleet  requirements) 

• Individual,  team,  or  group/ force  training  device. 

An  initial  effort  is  to  specify  the  fleet  requirements 
for  a training  system  ( shorebased  or  vehicle  based) 
and  the  personnel  involvement. 

Individual  Training- -emphasis  is  placed  on  need, 
type  of  training  and  aspects  of  the  operational 
world  in  which  training  is  accomplished,  for 
example , an  OFT  requirement  specifying  train- 
ing of  student  aviators  and  the  maintenance  of 
proficiency  of  rated  aviators  in  instrument  flight 
procedures  in  UH-1  rotary  wing  aircraft;  or  an 
OFT  specifying  aviator  flight  training  only  for  a 
dual  place  single  turbojet  engine  TA-4F  aircraft. 

Team  or  Group/ Force  Training- -emphasis  is 
placed  on  type  of  training,  for  example  , team 
coordination  procedures,  tactical  decision- 
making, extent  of  coverage  of  the  real-world 
events  . These  options  set  the  stage  for  decisions 
on  how  much  of  the  operational  world  to  represent. 

• Specific  sy  stem  vs  . univer  sal/ generalized  train- 
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The  key  decision  is  the  extent  of  the  use  of 
abstraction  vs . a high  degree  of  realism  in 
trainer  design,  for  example,  the  decision  to 
represent  realistically  the  compartments  and 
equipments  for  device  14A2,  ASROC/ASW 
Early  Attack  Weapon  System  Trainer,  or 
the  decision  for  greater  abstraction  (e.g.  , 
low-level  sensor  fidelity)  in  representing 
stations  in  device  14A6,  Coordinated  ASW 
Tactics  Trainer. 

Similar  decisions  are  made  concerning  the 
need  for  general/ universal  devices  for  in- 
dividual training. 

• Part  vs.  whole  system  training. 


The  decision  concerns  what  portion(s)  of  the 
equivalent  real-world  are  to  be  installed  for 
training . 


b.  Training  areas  to  represent  in  the  device  (trainee  station(s) 

design  philosophy) 

# The  decision  concerns  the  selection  of  a single  area 
(e.g. , single  aircraft  cockpit)  vs.  multiple  but 
interdependent  areas  (e.g.  , ASW  tactical  team 
training)  vs.  multiple  but  independent  areas  (e.g.  , 
multiple  independent  booths  for  aerial  navigation 
training) . 

• At  issue  is  the  definition  of  the  configuration  and 
the  operational  stations  to  represent.  This  in- 
volves also  the  initial  definition  of  the  trainee 
functions  to  be  performed  at  each  station. 

c.  Definition  of  system  components 

. This  concerns  the  decision  to  provide  specific 
system  suites /equipments  vs  . generalized/ non- 
specific equipments  . The  decision  is  amplified 
by  a discussion  of  the  rationale  for  the  alterna- 
tive selected. 


d.  Levels  of  training 

. Design  for  initial,  advanced,  transition,  or  main- 
tenance of  proficiency  training  is  determined. 
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Training  emphasis 

• For  single  trainee  situations --define  the  range 
of  the  job  training  to  be  accomplished  in  the 
device.  .• 

• For  multi-man  situations  (multiple  trainee  stations) 

the  design  alternatives  are  based  on:  a selective 

training  emphasis  (focusing  training  on  key 
positions  in  the  team  context  while  other  posi- 
tions are  only  supportive),  vs.  an  equal  training 
emphasis  (all  trainees  on  the  team/receive 
similar  training  emphasis)  ' : . 

• Definition  of  the  background  and  preparation  of 
the  trainee  population  is  accomplished,  describ- 
ing minimally: 

Who  the  trainees  will  be 

- Entry-level  capabilities  (extent  of  previous 

' experience  or  billet  qualifications) 

Training  exposure  (one  time  vs.  short  se- 
quence vs . recurring/ refresher  training) 
and  length  of  the  training  sessions  anti- 
cipated 

Implications  for  fidelity  of  simulation  (extent  of  simulation 
for  trainee  station(s) 

• This  ranges  from  high  fidelity  in  simulation  through 
deliberate  departures  from  realism  to  enhance 
training  value . The  requirement  is  to  identify 

the  simulation  requirements  in  fidelity  levels 

based  on  the  nature  of  the  training  desired. 

• The  initial  design  alternatives  are  based  on  the 
implications  of  the  performances  required  in  the 
device,  i.  e accounting  for: 

- Tactical  decision  making  vs.  procedures 
following  behavior 

Manual  control  requirements  Y_  : ; - . r 

• - . Displays  >•  Y. 

Communications 

' : yy' ■ ■•••■  ..."  . 
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• . The  design  decision  considers  the  implications  of 

classes  of  beihavior  to  be  trained  on  fidelity  re- 
quirements. High  fidelity  requirements  usually 
pertain  to  controls,  displays,  compartment  equip- 
ments, communications,  etc.  , involved  in  proce-: 
dural  and  manual  control  performances;-  lower 
fidelity  and  greater  reliance  on  instructor  medi- 
ation is  usually  pertinent  to  decision  making  and 
tactical  problem  solving  performances  . Thus, 
initial  decisions  on  fidelity  requirements  are 
made  here . in: terms , of  the  nature  of,  or  emphasis 
on,  the  classes  of  behavior  involved  in  training; 
the -design  issue  deals  with  the  extent  of  Simula - 
; tion(e.g.,  how  much  and  how.  well  the  representation) 

• Possibilities  for  stylized  or  innovative  techniques 

for  hardware/.software  are  identified  (this  may 
include  deliberate  departures  from  the  realism 
associated  with  the  operational  system  being 
simulated).  :.  r . 

• The  state-of-the-art  inadequacies  for  achieving 

the  tolerances  demanded  for  simulation  are 
identified.  ■»  . .. 

g.  Device  dynamics  (computer  programing  requirements) 

Gross  definition  of  the  characteristics  of  the  training 
environment  is  made  concerning:,  own  vehicle  and 
support  vehicles;  target( s)  characteristic s;  and  media 
characteristics.  (This  initial  definition  is  later  ex- 
panded in  Phase  5 where  the  .simulation  elements,  sub- 
sumed under  each  of  these  three  major  simulation 
categories  are  specified  together  with  the  number  of, 
and  desired  envelopes/ ranges /values  for  each.  ) 

Provisions  for  the  Management  of  Training  at  the  Instructor 

a.  Number  of  instructors  and  operators. 

b . Instructor  to  student  ratios  during  a training  exe r ci s e . 

c . Training  device  mode(s)  of  operation.  . 

The  decision  concerns  the  use  of  manually  controlled 
exercise  sequencing  vs . automated  (preprogramed 
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scenarios)  exercise  sequencing,  and  the  various 
modes  required  for  instructor  monitoring  and 
control  of  training. 

• Feasibility  of  employing  automated  adaptive 
training  modes  in  exercise  sequencing  is 
examined. 

d.  Pre-mission  requirements 

• Trainee  briefing  requirements  are  described.  ' 

, The  means  for  selection/ modification/ construction 
of  new  mission  scenarios,  off-line  (if  automated) 
are  described. 

e . Approach  to  monitor,  control  and  evaluation  of  training 

(enroute) 

• Flexibility  requirements  are  identified  for  the  display 
of  trainee  performance  information  and  control  of 
the  exercise  (manual  or  automated  scenarios) . 

• The  need  for  instructional  assists  via  hardware/ 
software  is  examined. 

f . Performance  monitoring/assessment 

• Measurement  system--the  decision  concerns  the 
need  for  a measurement  capability  in  the  device, 
and  whether  the  measurement  system  will  be 
automated  or  non-automated. 

• Requirements  are  set  for  the  display  of  trainee 
performance  information  (error  indications). 

. ‘ : • . ‘ . . : r 

. Decision  on  hard  copy  records  of  performance 
(for  critique  purposes  and  for  school  record- 
keeping  purposes)  are  made  as  to  form  and 
content. 

g.  Post-mission  requirements 

• Critique -hard  copy  printout  (quantitative  data  in 
form  and  content)  requirements  are  set. 


NAVTRADEVCEN  69 -C- 02 98-1 


. The  requirement  for  projection  display  equipment(s) 
for  mission/exercise  reconstruction  in  critique  are 
specified. 

h.  Configuration  of  instructor  station 


• Basic  console  requirements  are  outlined. 

• Decisions  on  the  use  of  repeater  displays  vs. 
non-repeaters  (e.g.  , CRTs)  are  made. 


. Any  special  layout  considerations/ design 
features  are  identified. 


2.1.2  Output.  An  initial  description  of  the  device  is  prepared,  defining 
what  the  training  device  will  be . The  articulation  of  the  purpose  of  the 
proposed  training  device  provides  an  initial  concept  of  the  device  and  a 
synopsis  of  the  overall  training  system.  This  "scoping  and  roughing  out" 
of  the  system  among  design  personnel  and  using  agency  personnel  provides 
the  focus  for  the  subsequent  iterations  leading  to  the  decisions  on  the 
functional  characteristics  that  define  the  desired  device.  The  same  issues 
that  are  identified  in  this  initial  analysis  are  further  defined  and  explicated 
in  the  subsequent  analyses. 

2.1.3  Summary,  Definition  of  Purpose  of  Training.  The  need  for  the 
careful  and  precise  delineation  of  the  purpose  of  the  training  device  cannot 
be  overemphasized.  The  final  decisions  on  what  must  be  simulated  and  the 
selection  of  design  from  among  alternatives  continually  tie-in  with  the 
definitions  of  the  purpose  of  the  training  device.  Total  design  is  most 
meaningful  when  specified  in  light  of  what  the  training  is  supposed  to 
accomplish. 


To  assure  the  optimum  output  in  this  phase,  it  is  mandatory  that 
all  individuals  involved  in  the  design  have  a common  understanding  of  the 
training  implications  for  the  device  in  question.  Using  a simulator  simply 
as  a substitute  for  operational  practice  is  contrary  to  the  philosophy  under- 
lying training  in  synthetic  environments.  The  purpose  of  such  training  is 
to  shape  relevant  behaviors  in  order  to  achieve  transfer  of  training  to  the 
operational  situation.  The  shaping  of  behavior  is  accomplished  most 
effectively  when  the  training  device  is  viewed  as  an  instructional  system. 

Simulation,  by  definition,  provides  precise  control  of  -the-  task 
situations  and  enables  deliberate  omissions  of  aspects  of  the  real  environ- 
ment that  are  not  critical  to  performance,  or  conducive  to  learning  or  . s 
which  do  not  yield  sufficient  training  advantage  for  costs  involved.  It  is 
thus  important  that  training  agencies  who  use  synthetic  training  systems 
be  encouraged  to  incorporate  relevant  training  concepts  in  the  specification 
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of  device  requirements.  These  concepts , discussed  in  this  document, 
subsume  the  following: 

• Achieving  fidelity  of  simulation  with  less  than  one-to- 
one  engineering  replication. 

• Deliberate  departures  from  realism  (thinking  about 
training  in  non- operational  environment  terms  and 
in  abstract  terms  as  well  as  in  terms  of  hardware 
duplication). 


• Consideration  of  design  in  terms  of  device  utilization, 
with  an  emphasis  on  the  structure  and  control  of  training; 
on  monitoring,  evaluation  and  scoring  of  performance; 
and  on  the  development  of  training  strategies . 

• Measurement  requirements  (display  and  recording)  for 
critique,  evaluation  and  quality  control  of  training. 

2.2  PHASE  2:  ORGANIZING  INFORMATION  ON  THE  OPERATIONAL 

SYSTEM.  The  statements  of  training  needs  developed  in  Phase  1 provide 
the  essential  outline  for  the  training  device  and  for  the  portions  of  the 
operational  system  on  which  information  will  be  assembled.  This  estab- 
lishes the  basis  for  the  types  of  training  required,  the  magnitude  of  the 
training  effort,  and  gross  concepts  about  the  fidelity  of  simulation  required. 

A detailed  familiarization  with  the  operational  system  is  now  begun, 
and  information  is  assembled  which  is  sufficient  and- necessary  for  defining  . ; 
task  structure.  What  is  required  is  an  understanding  of  system  objectives; 
system  structure,  flow  and  equipment  nomenclature;  and  system  capabili- 
ties. The  information  sources  consulted  include  the  following: 


Naval  doctrinal  publications  . . 

Relevant  technical  publications;  system  documentation.; 
Discussions  with  operational  system  experts. 

Observation  of  the.  operational  system  or  portions  thereof. 


. The  system  familiarization  should  center  on  an  adequate  descrip- 
tion of  the  operational  system  pertinent  to  the  projected  training  system 
and  on  an  understanding  of  the  man-machine  interactions . The  description  . - 
is  usually  sufficient  when  the  missions  performed  by  the  system  are  laid 
out,  together  with  the  functions  performed  by  personnel  who  will  be  involved 
in  the  training  system  under  consideration.  The  .under  standing  of  man-; 
machine  interactions  .usually  involves  a description  of  the  stations  (number- 
of  operators) , the  operator  dutiss , and  a depiction  of  the;  sequence  of 
operations  in  the  ays  tern.  (A  number  of  formats  are  available  for  repre- 
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sequence.  These  are  described  subsequently.) 
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A checklist  of  classes  of  information  to  organize  will  facilitate  this 
inquiry.  The  checklist  provides  information  on  the  description  of  the 
system,  system  organization,  equipment  and  flow,  and  man- machine 
interactions.  The  kinds  of  information  needed  are  shown  below  in  repre- 
sentative lists  of  questions  to  be  resolved  (see  Chenzoff  and  Folley,  1965). 

2.2.1  System  Description. 

• Is  this  a new  system  or  an  improvement  on  an  existing  system? 

• What  are  the  missions  of  the  system;  interactions  with  other 
systems  ? 

• What  specific  functions  does  it  perform? 

• What  tactical  concepts  (if  pertinent)  underlie  the  system? 

• Have  performance  standards  been  established?  ■ i. 

• What  are  the  functional  components  of  the  system  pertinent 
to  the  anticipated  training  system? 

. What  are  the  relevant  subsystems  and  their  interrelationships 
within  and  with  other  units? 

• What  are  the  inputs  to  the  system- -classes , typical  rates  and 
maximum  rates? 


• What  operations  does  the  system  perform  on  the  inputs? 

• What  are  the  required  outputs , where  do  they  go  and  in  what 
forms  ? 


• What  are  normal  and  minimum  acceptable  levels  of  system 
v > output,  what  is  the  maximum  system  capability  for  outputs? 

• Where  ire  the  points  where  the  quality  of  system  outputs 

• - can  be  evaluated?  ■;  ^ 


An  example  of  system  descriptive  information  for  ASW  destroyer 
operations  is  shown  in  Section  IV,  paragraph  4.2.1 ./ 

2 . 2.2  ‘ Man-Machine  Interactions  . 


What  are  the  human  operator(s)  roles  in  the  system? 


What  and  how  many,  personnel  are  involved  in  their  specific 
duties  ? /:  •• 
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. What  is  the  sequence  of  activities  during  a mission? 
• What  are  the  team  requirements? 


. What  sensory  inputs  are  received  by  operators,  how  displayed? 

. What  controls  are  operated,  what  are  the  outputs? 

. What  personnel  qualifications  are  pertinent? 

• Are  there  performance  standards  (e.g.  , speed,  accuracy) 
for  portions  of  system  operation? 

• What  situational  or  environmental  stresses,  and  safety  pre- 
cautions are  personnel  required  to  perform  under? 

. What  aspects  of  the  system  will  cause  special  difficulties 
for  training? 

The  outputs  of  this  effort  include: 

, A description  of  how  the  system  is  operated  and  employed 
with  emphasis  on  those  portions  of  the  system  pertinent  to 
the  planned  training. 

• Human  operator  roles  in  the  system  and  the  functions  performed 
by  each  individual  or  team. 


• Layout  of  the  system  by  functional  blocks,  and  analyses  of  the 
event  sequences  (actions,  decisions,  communications)  in 
system  operations. 

This  systems  familiarization  phase  should  be  accomplished  quickly 
as  a prelude  to  the  more  detailed  analysis  of  task  structure.  Several  - 
approaches  (outlined  in  2.2.3. 3)  are  employed  interactively  to  obtain  and 
organize  the  desired  information  for  describing  the  system. 

2.2.3  System  Documentation.  Use  is  made  of  available  documentation 
on  the  system  under  study.  These  sources  provide  an  overall  description 
of  the  system  components , flow,  operating  modes,  equipment  complexes, 
functions  performed  by  people,  and  so  forth. 


2.2.3.  1 Discussions  with  System  Experts.  The  most  productive  source 
of  systems  descriptive  information  is  obtained  from  operating  personnel 
who  are  subject-matter  experts  in  the  system,  and  every  effort  should  be 
made  to  meet  with  these  people  as  soon  as  possible  and  for  the  length  of 
time  required  for  system  familiarization.  * In  conjunction  with  this , every 
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opportunity  should  be  taken  to  observe  the  system  first-hand  and  receive 
"walk-throughs”  of  the  basic  operations  from  operating  personnel. 

2. 2. 3. 2 System  Layout.  Most  often,  operator -oriented  documentation 
such  as  personnel  functions  and  duties,  time  and  event  sequence  charts  of 
system  activities,  information  flow  charts,  etc.,  are  not  available.  Thus, 
the  human  factors  specialist  uses  graphic  analysis  techniques  for  the  needed 
understanding  of  system  functioning.  These  event- linked  diagrams  present 
the  continuity  in  system  flow  (actions,  decisions,  communications)  and  the 
relationships  between  men  and  equipment.  The  operational  Sequence 
Diagram  (OSD)  is  representative  of  this  type  of  technique.  Ideally,  the 
development  of  this  graphic  analysis  should  be  accomplished  in  conjunction 
with  personnel  who  are  expert  in  the  system  operations  in  order  to  maxi- 
mize completeness  and  accuracy,  and  to  shorten  the  time  to  accomplish. 

Thus,  as  mini  mum  s , the  following  classes  of  information  are 

organized: 


• Information  on  the  missions  performed  by  the  system. 
This  is  prepared  in  a tabular  descriptive  format. 


• Functions  performed  by  individuals  and  by  subteams 
(as  pertinent)  in  the  system.  This  is  prepared  in  a 
tabular  descriptive  format. 

• Information  on  duties  performed  by  each  relevant 
operator  and  by  duty  stations  (as  pertinent)  for  the 
system.  This  is  prepared  in  a tabular  descriptive 
format. 


• Identification  of  system  suites  and  equipment  com- 
ponents (as  pertinent).  In  the  case  of  universal/ 
generalized  trainers,  specific  equipment  require- 
ments are. identified  where  replicas  of  operational 
equipments  are  needed,  and  the  range  of  mission 
functions  and  events  to  be  simulated  are  identified 
where  non- specificity  in  equipments  are  called  for. 


• Sequence  of  system  operation  and  functional  layout 
of  the  system.  Various  graphic  formats  are  avail- 
able for  gaining  a gross  understanding  of  the  inter- 
actions of  men  and  equipment. 

2.2. 3.3  Graphic  Analysis  Formats.  Graphic  analyses  are  employed  to 


logically  reduce  a complex  system  or  operation  to  a clearly  interrelated 
set  of  functions,  tasks  and  activities,  e.g.  , a simplified  presentation  via 
diagrams  of  a complex  series  of  relationships . • 
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A number  of  variations  in  technique  are  available . For  single 
operator  systems.  Information,  Decision,  Action  (IDA)  charts  are  useful. 

The  format  for  an  IDA  diagram  and  typical  symbols  used  in  analyzing  man- 
machine  operations  is  shown  in  Figure  1.  For  multi-man  operations , a 
relatively  simple  multiple -activity  process  chart  may  be  used.  Figure  2 
provides  an  example.  In  practice,  the  most  prevalent  format  for  describ- 
ing system  functions  and  man-machine  interactions -is  the  Operational 
Sequence  Diagram  (OSD).  It  is  structured  around  defined  pieces  of  hard- 
ware, software,  and  operators,  and  is  used  to  identify  their  interrelation- 
ships as  the  system  is  exercised  against  a scenario.  Unfortunately,  no 
single  graphic  format  (classes  of  information  and  symbolic  logic)  has  been 
standardized.  In  its  simplest  form  the  OSD  is  used  to  establish  a sequence 
of  operations  required  between  subsystem  interfaces  at  various  levels  of 
operation.  In  some  instances,  sequence  diagrams  are  similar  to  block 
diagrams  of  a system,  with  each  block  in  the  series  describing  gross  as- 
pects of  system  operations.  Sequence  diagrams  can  be;  laid  out  in  various 
ways.  They  may  be  spatially  oriented  as  well  as  placed  along  a time  line , 
or  they  may  be  used  simply  as  an  ideographic  convenience.  The  constant 
purpose,  however,  is  to  organize  system  information  in  a way  to  depict 
the  significant  events  and  activities  in  operating  the  man-machine  system 
being  analyzed.  Generally,  a columnar  format  with  a vertically  descend- 
ing time  line  is  most  preferred.  Within  this,  the  OSD  is  developed  as  a 
working  document  according  to  the  requirements  of  the  particular  job 
objectives. 

An  example  of  the  original  development  by  Dunlap  and.  Associates 
Inc.,  (Brooks,  I960)  is  shown  in  Figure  3 . Four  symbols  are  used  to  de- 
note operator  action,  and  decision  and  information  transmitted  or  received. 
Double-lined  symbols  indicate  automation  of  the  event.  Usually,  numbers 
inside  symbols  are  used  for  reference  to  a key.  This  requirement  for 
cross-referencing  may  become  cumbersome  in  lengthy  operational  se- 
quences. Thus , a variation  is  seen  occasionally  whereby  the  information 
is  written  within  the  symbol  thereby  eliminating  the  need  for  cross- 
referencing.  Another  example  is  shown  in  Figure  4 based  on  the  symbol 
code  specified  in  Navy  Specification  MIL-H-24 148  (SHIPS  1965). 

Wilson  (1966)  has  attempted  to  standardize  OSD  symbols  by 
selecting  tbiose  most  commonly  used.  Six  symbols  are  recommended 
together  with  modifying  indicators  which  make  them  applicable  to  almost 
any  OSD  usage.  These  are  shown  below  in  Figure  5 . An  example  of  an 
OSD  using  this  format  is  shown  in  Figure  6.  : Additional  information  on 
OSD  employment  and  format  is  found  in  NAVWEPS  18413A  ( 1963)  . 

2.2.4  - Output.  The  effort  thus  far  enables  a number  of  device  design 

decisions  to  be  made.  These  include: 

. Decision  on  the  level  of  abstraction  to  be  achieved  in  the 
0 trainee  station  displays  and  controls . 

ERIC" 
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Figure  1.  An  IDA  Diagram  and  Typical  Symbols  as  Used  in  Analyzing  Man-Machine  Task  Performance 
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TASK:  Mating  of  a Rescue  Submersible  With  a Downed  Submarine  (Distress) 


(Previous  Task  Accomplished:  Approach  to  Distressed  Submarine 

and  Sensor  Contact  With  Vehicle) 
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Figure  2.  Multiple  Activity  Process  Chart  (Part  1 of  2) 


NAVTRADEV CEN  69-C-0298-1 


Jmmutes) 


PILOT 


CO-PILOT 


CREWMAN 


23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

42 

43  i 


Maintains 

Position 


Monitors  audio  and 
video  sensors 


Activates  ana  cnecJks 
out  mating  sensors 
(lights  , viewing  op- 
tics , TV,  mating  hatch  sonar 


Clears  hatch  area  of 
debris  with  water  jet 
and  brush,  cuts  mes  - 
senger  buoy  cable 


Prepares  for  final 
descent  and  mating., 
deploys  pre-mating 
subsystems 


Positions  DSRV 
on  hatch  of 
submarine 


Stabilizes  DSRV  for 
final  mating,  effects 
final  airtight  seal 
to  hatch 


Verifies  mating  of 
DSRV  with 
submarine  (hull 
pressure) 


Monitors 

all 

displays 

and 

reports 

to 

pilot 


Inspects  visually  the 
mating  area  through 
viewport  and  reports 
to  pilot 


Figure  2.  Multiple  Activity  Process  Chart  (Part  2 of  2) 
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Figure  3.  An  Example  of  an  Operational  Sequence  Diagram 


MISSION  SEGMENT:  SECOND  RECON.  TARGET  RUN  ■■  OPERATOR  POSITION:  PRPV 
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Figure  4.  Operational  Sequence  Diagram  Format  (Part  1 of  2) 
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BASIC  FUNCTIONAL  SYMBOLS 


o 

o 

Transmission  of  Information 

O 

Previously  Stored 
Information 

Receipt  of  Information 

o 

Decision 

V 

Storage  of  Information 

□ 

Action 

MODIFYING 

INDICATORS 

When  necessary  to  specify  machine  behavior,  add  a line  across  the  bottom 
of  the  symbol,  thus: 


Human  Action 


Machine  Action 


When  necessary  to  indicate  storage  of  information  on  display  to  humans, 
insert  a "DM  indicator  inside  the  "store"  symbol,  thus: 


w 


Store  information  (manually  as  on  a plot  board) 

Store  information  (automatically  as  on  a radar  scope,  indicator 

light,  or  readout) 


When  necessary  to  indicate  ah  inverse  meaning,  add  a diagonal  line  across 
the  symbol,  thus: 


□ Action  (yes,  "go") 


Inaction  (no,  "no-go") 


When  necessary  to  specify  means  of  communication,  insert  an  appropriate 
indicator  inside  "transmit"  and  "receive"  symbols,  such  as: 


E Electric  or  Electronic 

V Visual  SP  Sound -powered  Phone 

S Speech  T Touch 


Figure  5.  Suggested  OSD  Symbology 
(from  Wilson  , 1966) 
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Figure  6.  Example  of  Detailed  Functional  Analysis  OSD 
(from  Wilson , 1966) 
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• Definition  of  training  equipment  requirements:  system 

suites,  equipment  components  for  specific  systems;  or 
generalized  components  to  provide  for  the  range  of 
mission  requirements  for  universal/ general  training. 

• Definition  of  key  system  components. 

• Decisions  on.  what  components  can  be  omitted  and  on 
those  components  that  are  not  crucial  to  training. 


Description  of  key  personnel  positions. 

• Definition  of  the  range  of  mission  activities  to  be 
accounted  for  in  the  device. 

These  analyses  are  the  prelude  to  the  analysis  of  human  involve- 
ment in  the  training  system  and  to  the  overall  definition  of  the  device  con- 
figuration. The  importance  of  this  effort  must  not  be  minimized,  for  these 
activities  provide  a vital  input  to  later  analyses  on  design  decisions  leading 
to  the  specification  of  the  functional  characteristics  of  the  operational  en- 
vironment to  simulate  for  training. 

2.2.5  Summary:  Organizing  Information  on  the  Operational  System. 

Becoming  familiar  with  the  operational  system  as  a prelude  to  training 
device  design  is  a simple  and  meaningful  statement.  No  one  would  contest 
the  logic  that  a fundamental  understanding  of  the  system  in  question  and 
the  kinds  of  training  problems  involved  provides  the  groundwork  for  device 
design  decisions . The  manner  of  achieving  this  is  something  else.  What 
and  how  much  information  is  required  and  how  difficult  is  it  to  obtain  and 
organize  the  information,  are  the  paramount  questions.  The  answer  is 
simple  when  in  the  form,  of  a principle:  gather  that  information  which  will 

provide  a correct  picture  of  which  subsystems  have  the  major  influence  on 
the  total  system.  In  other  words,  organize  only  the  information  which  en- 
ables an  evaluation  of  what  operator  tasks  have  high  skill  requirements,  are 
importantly  related  to  system  effectiveness,  and  can  yield  improved  per- 
formance through  training . In  essence,  what  is  needed  is  information  in 
a form  which  will  facilitate  determining  the  importance  of  task  perform- 
ance to  system  effectiveness  . 

Before  we  succumb  to  the  lure  of  the  graphic  analysis  techniques, 
let  us  reiterate  that  the  purpose  of  these  analytic  efforts  is  to  gain  the 
system  familiarization  necessary  to  specify  the  characteristics  of  the  syn- 
thetic training  system  under  consideration,  no  more,  no  less.  The  OSD 
is  recommended  only  to  facilitate  the  understanding  of  the  basic  processes 
in  system  composition.  . The  decision  to  do  an  OSD  tends  to  insure  an 
understanding  of  a system,  in  that  the  OSD  structure  demands  a continuity 
in  system  flow,  and  establishes  relationships  between  operators  and 
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equipment  and  between  operators  and  tasks.  It  also  serves  as  a visible 
basis  for  confirmation  of  the  system  description  with  operational  personnel. 
To  the  extent  that  information  on  human  involvement  in  the  system  is  avail- 
able (e.g.,  standard  procedures,  information  flow  sequences,  operator 
functions  and  duties)  then  graphic  techniques  may  not  be  required.  (Typi- 
cally, this  is  not  the  case  since  system  documentation  most  often  centers 
on  equipment  capabilities).  OSD's  are  tailored  to  a specific  requirement 
since  a complex  training  device  is  a.  one-of-a-kind  or  at  most,  a few-of- 
a-kind  endeavor.  Although  there  are  commonalities  within  classes  of 
systems,  the  analysis  is  specific  to  the  given  system  and  is  expensive  in 
time  and  effort.  Thus  , the  system  familiarization  phase  must  be  organized 
to  gain  the  necessary  information  most  economically . This  means  that  the 
need  for  elegant  analyses  must  be  justified  in  light  of  available  information 
sources. 


2.2.5. 1 Research  Issue.  The  issue  of  how  much  information  is  required 
and  in  what  form  for  the  system  familiarization  stage  is  not  easily  resolved. 
Available  graphic  techniques  are  used  for  various  system  design  purposes. 
The  OSD,  for  example,  is  used  in  prime  system  design  to  establish  system 
requirements,  to  allocate  man-machine  functions,  to  determine  sequences 
of  operations,  and  to  evaluate  equipment  and  panel  layouts  and  the  design 
of  workspaces.  It  is  also  used  in  personnel  research  and  in  training  sys- 
tem design  and  utilization,  although  the  purpose  and  emphasis  is  different 
in  each  case.  Yet,  OSD  guidelines  and  formats  are  presented  as  if  the 
method  is  invariant  for  any  purpose.  Guidelines  on  uses  of  analytic  tech- 
niques relative  to  training  requirements  determination  are  needed,  speci- 
fically, the  uses  of  graphic  analysis  in  the  design  of  training  systems  (when 
to  employ,  how  much  and  what  kind  of  information  to  organize,  and  how  the 
information  is  used  as  a base  for  further  analysis  of  operator  tasks). 


2.3  PHASE  3:  ANALYSIS  OF  TASKS  INVOLVING  HUMAN  PARTICI- 

PATION. Preparation  has  now  been  completed  for  an  analysis  of  the  tasks 
that  people  perform  in  the  operational  system,  in  order  to  define  task 
structure  relative  to  decisions  about  training.  Task  information  provides 
a basis  for  determining  whether  or  not  training  equipment  is  indeed  nec- 
essary to  support  a training  effort  in  the  operational  context  (e.g.  , deci- 
sions on  the  need  for  a training  device-vs.  the  suitability  of  simple  training 
aids  for  achieving  defined  training  purposes).  In  instances  where  a train- 
ing device  need  is  justified,  the  development  of  the  functional  requirements 
for  the  simulator  is  based  significantly  on  determinations  of  what  the  indi- 
vidual s)  must  do  in  the  operational  situation.  Thus,  an  understanding  of 
the  kinds  or  types  of  training  to  be  accomplished  is  a straightforward  re- 
quirement in  the  design  process.  The  understanding  of  the  training  problem 
and  the  identified  design  pathway  set  the  guidelines  for  the  analysis  of  tasks; 
the  system  information  provides  the  base  from  which  the  task  analysis  is 


derived. 
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The  effort  in  this  phase  is  geared  to  identifying  and  organizing  the 
activities  involved  in  job  performance  to  enable  decisions  about  character- 
istics/of  the  training  system.  This  is  done  at  a level  of  detail  which  will 
assist  in  the  identification  of  design  alternatives  to  achieve  the  purpose  of 
training.  Nothing  more  than  this  is  needed. 

Decisions  on  training  system  design  are  made  in  successive  stages. 
Correspondingly,  system  and  task  information  is  gathered,  organized,  and 
refined  in  a like  manner.  Two  classes  of  requirements  are  accomplished 
in  this  phase:  the  definition  of  the  operational  task  structure;  and  the  defi- 

nition of  training  objectives. 

2.3.1  Operational  Task  Structure . An  effective  task  structure  analysis 
should  have  a training  hardware  orientation  with  emphasis  on  the  man- 
equipment  interfaces.  It  is  accomplished  in  sufficient  detail  to  identify 
design  options  available  to  meet  the  purpose  of  the  training  system.  The 
hardware  orientation  is  necessary  also  to  insure  that  the  functional  design 
selected  has  the  flexibility  to  accommodate  needed  changes  in  the  training 
system  that  inevitably  arise  from  experience  with  the  device  (including 
changes  in  operational  doctrine,  etc.),  once  it  is  in  operation.  To  be 
avoided  is  the  exercise  of  doing  an  analysis  which  is  lengthy  and  time- 
consuming  and  elegant  in  format,  but  which  provides  unnecessary  informa- 
tion for  translating  task  description  data  into  a set  of  functional  specifica- 
tions for  device  design. 

Task  organization  is  accomplished  in  operational  terms  — what 
each  trainee  is  accountable  for  across  the  range  of  training  missions /events 
The  analysis  is  conducted  only  to  the  task  level  which  provides  the  informa- 
tion necessary  for  training  decisions.  Most  often,  further  refinements  in 
detail  are  extravagant  of  time  and  effort  and  contribute  very  little  more  to 
the  design  decision  process. 

Decisions  on  training  system  design  are  made  in  successive  stages 
correspondingly,  system  and  task  information  is  gathered,  organized  and 
refined  in  a like  manner.  Task  analysis  progresses  in  stages,  undergoing 
continual  refinement  to  achieve  the  definition  of  task  structure  required 
for  making  training  design  decisions. 

The  starting  point  is  the  system  descriptive  information  and  the 
defined  equipment  complexes  of  the  operational  system.  At  this  point  in 
the  design,  cycle  the  following  has  already  been  determined. 

• Decision  on  the  level  of  abstraction,  i.e.  , the  realism 
to  portray  in  the  training  system. 
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• Definition  of  system  suite--those  complexes  of  equipment 
to  be  duplicated  in  the  trainer  (and  those  related  sub- 
systems that  are  not  relevant  to  the  device  under 
consideration). 

• Decisions  on  key  system  components  with  dynamic 
fidelity  implications. 

• Decisions  on  what  functions  of  the  relevant  operational 
system  are  to  be  omitted. 

« Decisions  on  what  components  of  the  training  system 
are  not  crucial  to  the  training,  but  serve  to  support 
the  training  and  thus  are  included  in  design  (secondary 
emphasis). 

• Description  of  the  key  personnel  positions  in  the  training 
system. 

• Definition  of  the  range  of  mission  activities  to  be  included 
in  the  training  system.  For  example,  in  ASW  operations, 
it  could  involve  one  and  two  destroyers  in  screen  and  SAU 
operations,  weapon  delivery  of  ASROC , torpedoes  and 
DASH;  coordinated  aircraft  operations  involving  VECTAC's, 
etc . 

The  effort  begins  with  a list  of  positions  in  the  system  and  a 
description  of  the  operational  tasks  pertinent  to  each  position.  The  pro- 
gression for  developing  the  task  inventory  starts  with  an  individual  position 
(and  where  applicable,  the  subsystem  in  which  it  is  imbedded)  and  defines 
the  functions  performed,  then  the  task  areas  , and  finally  the  operational 
activities  in  each  task.  Information  is  gathered  from  various  sources: 
observing  tasks  performed  by  job  incumbents:  reviewing  documents;  and 
discussions  with  subject  matter  experts. 

Thus , the  task  analysis  is  in  the  systems  context  and  only  for 
that  portion  of  the  system  that  is  to  be  represented  in  the  device  . The 
analysis  .should  yield  a listing  and  grouping  of  task  statements  to  provide 
a structure  which  defines  the  performances  to  be  accomplished  in  the  train- 
ing system.  The  following  information  is  organized  ii.  e .,  as  minima): 

• The  activity  elements  which  make  up  the  task. 

• The  trainee(s)  involved  in  the  activity. 

• Equipments  required  (displays  , controls , . communications)  . 

• Skills  and  knowledges  required  in  performance. 

• Any  unusual  conditions  of  performance. 
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2.3.  1.1  Representative  Task  Analyses  . There  is  considerable  variation 
in  task  analysis  techniques,  in  part  resulting  from  level  of  detail  and  em- 
phasis and  in  part  due  to  the  specific  design  situations  to  which  they  are 
applied.  The  most  common  task  analysis  techniques  employ  tabular  for- 
mats which  identify  some  or  all  of  the  following:  job  task  segments;  per- 

sonnel breakouts  (in  multi-man  situations);  equipment  involved;  conditions 
of  performance;  and  skills  and  knowledges  required  in  task  performance. 
Commonly,  the  activities  are  presented  in  a normal  or  logical  sequence  of 
occurrence  (see,  for.  example,  the  format  used  in  Section  IV  of  this  report 
for  destroyer  ASW  activities).  Time  information  may  be  implied  or  pro- 
vided by  a time  scale  or  by  statements  of  time  involvement.  Considerable 
numbers  of  task  analyses  have  been  conducted  during  the  last  two  decades 
and  much  published  evidence  of  the  technique  exist.  Several  sources  in 
the  literature  are  identified  here  to  provide  the  reader  with  a cross-section 
of  the  thinking  about,  and  application  of  task  analysis  techniques  (see,  for 
example.  Miller  ( 1962)  , Cotterman  ( I960)  , Smith  (1965),  Run  dqui  st  ( 1967  , 
1970),  NAVWEPS  (OD18413A)).  Since  task  analysis  techniques  and  infor- 
mation content  are  so  varied,  representative  analyses  are  shown  below  to 
acquaint  the  reader  with  a range  of  techniques  used  and  with  an  idea  of  the 
effort  that  must  be  expended.  The  examples  have  been  selected  to  empha- 
size the  differences  in  various  task  analysis  techniques. 

2.3.  1.1.1  Chenzoff  and  Folley --The  method  of  Chenzoff  and  Folley 

(1965)  is  an  attempt  to  systematize  task  information  relative  to  the  design 
of  training  devices.  This  task  analysis  method  was  designed  as  a major 
stage  in  the  Training  Situation  Ana.lysis  ( TSA)  process  developed  by  the 
Naval  Training  Device  Center  (Bertin,  1965)  for  defining  the  specifications 
for  training  support  equipment.  The  Chenzoff  and  Folley  method  develops 
a progression  from  gross  to  specific  task  information.  The  stages  in  the 
task  analysis  and  the  kinds  of  data  obtained  are  summarized  below. 

Stage  Analysis  Unit  Data  Obtained 

Total  system  Major  system  operations 

arranged  in  sequence  ?.nd 
type 

Task  time  charts  Operating  stages  Task  identification  and 

of  the  system  relationships  among 

(or  the  total  tasks 

system) 

Functional  task  Tasks  Activities  within  tasks  and 

descriptions  relationships  among 

activities 

Behavioral  details  Activities  with-  Detailed  characteristics  of 

descriptions  in  tasks  activities 


System  block 
analysis 
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a.  System  Block  Analysis 

System  tasks  can  be  identified  and  listed  directly  or  a 
block  analysis  can  be  accomplished  which  is  simply  a sequential  listing  of 
major  blocks  of  tasks  or  system  operations  into  which  a system  can  be 
partitioned. 

Example: 


Block 

Remarks 

1.0 

Accept  and  inspect  weapon 

2.0 

Assemble  weapon 

Follows  Block  1 

3.0 

Check  out  weapon 

Follows  Block  2,  9 or 

4.0 

Position  weapon 

Follows  Block  3 

5.0 

Check  assigned  targets 
within  range 

Follows  Block  4 

6.0 

Establish  Ready-to-Fire 
condition 

Follows  Block  5 

7.0 

Prepare  to  fire  and  fire 

Follows  Block  6 

8.0 

Travel  to  target 

Follows  Block  7 

9.0 

Scheduled  maintenance 

On  regular  schedule 

10.0 

Unscheduled  maintenance 

As  required 

b.  Task  Time  Charts 

In  this  stage,  the  tasks  are  identified,  the  time  and  coor- 
dination requirements  for  the  tasks,  and  anv  adverse  conditions  surround- 
ing performance  are  established.  In  the  accompanying  task  time  chart. 
Figure  7,  each  task  and  the  operator s are  listed;  time  lines  are  shown  to 
indicate  the  period  of  time  consumed  by  each  task.  Coordination  between 
and  among  people  is  shown  by  diagonal  arrows  and  wavy  lines  (a  dashed 
line  indicates  intermittent  coordination).  Any  adverse  conditions  (i.e.  , 
environmental  or  situational  factors  which  degrade  task  performance)  are 
also  recorded.  A - 
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c . Functional  Task  Description 

The  activities  and  interrelationships  within  tasks  are  de- 
fined in  this  stage.  Determined  here  are  the  time  to  perform  tasks,  the 
activities  within  tasks,  the  proportion  of  attention  each  activity  requires  of 
the  operator,  the  contingencies  (i.e.  , things  that  may  disturb  normal  per- 
formance such  as  equipment  malfunction,  human  error,  unusual  external 
situations)  and  any  adverse  conditions  that  apply  to  an  individual  task.  Six 
classes  of  task  activity  are  identified:  procedure  following,  continuous 

perceptual- motor  activity,  monitoring,  communicating,  decision-making 
or  problem-solving,  and  non-task  related  activity.  Figure  8 provides  the 
functional  task  description  form. 

d.  Behavioral  Details  Description 

The  most  detailed  level  of  task  description  is  developed 
at  this  stage  of  the  ana.lv  i.s.  Estimated  here  are:  trainee  capability  to 

perform  the  tasks,  the  d’fficulty  of  the  training  problem  associated  with 
each  task,  and  the  lev*?,  of  performance  that  can  be  expected  after  training 
(see  Figure  9). 


Chenzoff  and  Folley  believe  this  approach  will  enable  a 
precise  description  of  the  relevant  tasks  in  a system  and  will  provide  the 
information  necess-  ry  to  make  base  decisions  about  the  gross  features  of 
the  ultimate  training  program  (i.e.  , the  Functional  Training  Requirements 
phase  in  the  TSA  process  mentioned  earlier). 

2.3.  1.  1.2  Goodyear  - - Another  example  is  the  technique  employed  by 

the  Goodyear  Aerospace  Corporation  in  determining  Submarine  Casualty 
Control  Training  Requirements  (Goodyear,  1966).  This  study  determiued 
the  critical  factors  to  be  simulated  for  team  training  in  submarine  casualty 
control  by  an  analysis  of  crew  position  responsibilities  and  a task  analysis 
of  submarine  casualty  sequences.  Table  1 shows  an  example  from  the 
training  requirements  analysis  and  crew  position  responsibilities;  Table  2 
provides  an  example  from  a sequence  and  task  analysis;  and  Table  3 defines 
critical  factors  and  the  skill  and  knowledge  requirements. 

2.3.  1.1.3  Part- Task  Analysis --Task  Analyses  are  also  conducted 
specific  to  part-task  requirements  in  the  total  job  structure.  Table  4 
provides  an  example  which  is  concerned  with  determining  visual  and  motion 
cue  requirements  for  tasks  that  must  be  simulated  in  a tactical  flight  train- 
ing device  ( AH- 5 6A  helicopter)  . 

2.3.  1.  1.4  Siegel — Another  approach  to  defining  training  requirements 

was  developed  by  Siegel  and  Federman  (1970)  for  analyzirg  the  tasks  in- 
volved in  the  tactical  coordinator  function  in  the  P3C/VSX  aircraft.  Task 
stx-ucturing  was  accomplished  by  the  technique  of  intellective  load  analysis 
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?o“rTP»r_.  Eir*  f 7.  7) 

(Action  verb  and  object) 

Time  Performance  Requirement:  20  

Po  s i t i on  /’"/Ve  Co  ntro/  Supervisor- 

Using.  _ S u p r %/ i S Or* ConTrot  * ie/. 

In  rercgw 

(Equipment,  tools  or  other  materials) 


Project / OOP 

Typical  Task  Time  / & m * n vt"G  S 

Informant S m / 't  h 

Analyst ^ ° n & §_ 

Date  6 Q<zc  65 


Activity 

Percentage 
of  Attention 

Time  Relationships* 

Procedure  Following 

30 

& 

(3) 

Continuous  Perceptual 
Motor  Activity 

O 

Monitoring 

JO 

(1) 

<?> 

Communicating 

,° 

(K> 

Decision  Making  or 
Problem  Solving 

SO 

Other  (explain  in  notes) 

0 

Non-Task-Related 

Activity 

o 

— 

— 

Proportional  Time  Q .1  .2  .3  .4  .5  .6  .7  .8  .9  T 


Contingency 

I Cue 

Response 

Frequency 

Reference** 

Natch  Stuck 

f?e  </  A w/i  y 

See  Unsch&A.  Ma/nt 

.05 

JO.  3 

Computer  No-Go 

COMPUTER 

£ra  sed  /? e.  F~i r e. 

JO 

— 

d-o  m p u Zb  r 

Re.p^a.  V 

Ro-A  COMPUTER 

. 

By  - Ross 

.03 

/O.  4 

* * If  response  is  complex,  cross-reference  here  to  task  created  by  contingency. 


Adverse  Condition 

j Ssvarity 

Prob.  of 
Occurence 

% of  Time  or 
Prop.  Limits 

Crowding  in  on  standing  spac.e. 

1 2 

.Z5 

Z5% 

* 

*Show  initial  and  terminal  Activity  events  on  the  Time  Relationships  chart.  Explain  on  the  back. 

Figure  8»  Functional  Task  Description  Form 
(from  Chenzoff  pnd  Folley^  1965). 
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BEHAVIORAL  DETAILS  DESCRIPTION 


BLOCK  NO, 


•7.  £> 


TASK  NO, 


TTjL 


Procedure  Following 


-i^Fixed 
Branched  — 


Number  of  steps  in  procedure  Z-_ O 
Maximum  number  of  steps  in  procedure 


Number  of  SB  steps 2 


Number  of  possible  steps  which  are  SB  . 


Describe  below  the  kinds  of  SB  steps  involved: 

If  ^ffMputer  lights  red  af-ter  Erase  and  Citims-tes 

flue  tuatioru 


nee 


u U 


Continuous  Perceptual-Motor  Activity  1 

None 


Iype 

Guiding  a vehicle 

Operating  remote 

manipulators 

Keeping  cursor  on 

target 

Other  


Displays 

Direct  or  window  view 

Scope  or  instruments 

Optical  system 

Other 

Error  tolerance  or 
accuracy  required: 


Controls 

Steering  wheel 

Tracking  handle 

Handwheels 

Others 


Control-Display 

Relationship 

Position  control 

Velocity  control 

Acceleration  control 

— Lag 

Backlash 

Other 


Monitoring 


Object  or  signal  to  be  monitored: 


. L-LJ.gJLVs.- 


Display 

Scope 

Window  view 

^^Instruments 

Optical  system 

Sounds 

Other 


Relevant  Attribute 

Movement  of  object  or 

signal 

Appearance  of  object  or 

signal 

^IChange  in  object  or 
signal 

Other 


Other  Date 

Estimated  frequency  of  events: 

Search  area: 

Are  events  easy  to  detect? _ y & < 

If  "no",  how  should  detection  be  made? 


Communicating 


\ y Radio  or  telephone 

Direct  verbal 

Direct  observation 

Written  or  printed 

English 


Media 

Video 

Electro-mechanical 

displays 

Other 


Spec ial  Knowledge  Requirements 

— Code  Format 

Keyboard  operation 

— .Operation  of  special  equipment 
(Specify) 


e_ 


Decision  Making  and  Problem  Solving 


The  operator  considers  only  one  of  several  available  courses  of  action.  He  bases  his  action  on  a 

rule-of-thumb,  special  knowledge  or  experience,  or  memory  of  what  action  has  proven  successful  in  the  past. 

Reasonable  alternatives  are  generated,  considered,  and  rejected,  until  an  acceptable  one  is  found. 

-V^Most  possible  alternatives  are  known  by  the  decision  maker  or  problem  solver,  and  all  reasonable  ones  are 
evaluated. 


Describe  what  the  decisions  or  problems  consi' t of  and  the  kinds  of  information  used  in  reaching  the  decision  or  solution: 

(3a.S't  gighsl  G low  ~tUrou.gh  parallel  Select  and  Target 
C A a t'tr  e I St 

Figure  9.  Behavioral  Details  Description  Form. 

(from  Chenzoff  ?nd  Folley,  1965) 


- 38  - 

43 


NAVTRADEVCEN  69-C-0298-1 


TABLE  1.  TRAINING  REQUIREMENTS  ANALYSIS  AND 
CREW  POSITION  RESPONSIBILITIES 


I - CASUALTY(IES):  PLANE  AND  RUDDER  FAILURES 

A - PERSONNEL:  HELMSMAN,  STERN  PLANESMAN,  DO,  OOD,  COW, 
OFF-LOOKOUT  (OR  LEE  HELMSMAN) 

B - PART  TASK  GROUPING 

1.  HELMSMAN  RESPONSIBILITIES 

The  helmsman  responsibilities  are: 

1.  Select  control  mode. 

2.  Select  course  and  depth  on  AMC  control  panel. 

3.  Ring  up  ordered  engine  bells  and  acknowledge  orders. 

4.  Control  (normal)  rudder  to  gain  and  maintain  ordered  course. 

5.  Control  (normal)  fair  water  planes  at  moderate  to  shallow  depth  to 
reach  and  maintain  ordered  depth. 

6.  Monitor  indicators  to  ensure  correct  operation  of  indicators,  fair- 
water  planes , and  rudder  and  to  ensure  correct  operation  of  the 
ship: 

Combined  instrument  unit  indicators  are  gyro  course,  rudder  angles, 
depth,  trim  angle,  depth  error,  depth  rate,  and  course  error. 

Other  indications  are  deep  depth  gage,  shallow  depth  gage,  digital 
depth  indicator,  speed  indicator,  engine  order  indicator,  gyro 
course  indicator,  fairwater  planes  angle  (normal  and  emergency), 
rudder  angle  (normal  and  emergency),  and  Magnesyn  Indicator. 

7.  Notify  DOOW  of  improper  indications  or  failure. 

8.  Test  periodically  plane  and  rudder  control  (both  normal  and  emer- 
gency modes). 

9.  Switch  plane  and  rudder  control  to  emergency  power,  activating 
power  transfer  pilot  valves  (or  700  psi  air  in  case  of  594  class 
ships)  if  required. 
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TABLE  1.  TRAINING  REQUIREMENTS  ANALYSIS  AND 

CREW  POSITION  RESPONSIBILITIES  (Continued) 


10.  Test  and  operate  planes  in  rate  control  (emergency  power). 

11.  Request  the  lee  helmsman  to  man  emergency  steering  station. 

12.  Request  lineup  of  hydraulic  system  for  emergency  positioning 

pump  operation:  (a)  fairwater  planes  - lee  helmsman,  (b)  stern 

planes  and  rudder  - aft  auxiliary  electrician  on  phones;  engine 

; room  upper  level  watch  or  machinery  watch  supervision  on 
controls. 

13.  Exercise  combined  control  of  steering  and  planes  using  single 
stick  (optional  task). 

14.  Open  or  close  valve,  as  appropriate,  to  operate  or  isolate  shallow 
depth  gage.  Depth  limit  for  safe  gage  operation  is  about  200  ft. 

15.  Acknowledge  and  report  fulfillment  of  orders  by  OOD  and  DO. 

16.  Operate  fairwater  planes  to  compensate  for  stern  plane  failure 
or  as  directed  by  the  DO. 

(Responsibilities  are  also  defined  for  the:  stern  planesman,  duty  officer, 

OOD,  Lee  Helmsman  and  COOW.) 


(from  Goodyear,  1966) 
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TABLE  2.  SEQUENCE  AND  TASK  ANALYSIS 


C - FAIRWATER  PLANES  FAIL  ON  DIVE 

1.  RECOGNITION 

a.  Initial  Conditions  - Diving  officer  (DO)  and/or  planesmen  are  cog- 
nizant,of  ordered  and  actual  speed,  depth,  depth  rate,  attitude,  turn 
rate,  trim,  and  hydraulic  system  status. 

b.  Detection 

(1)  Detection  of  Impending  Casualties  - BCPO  notes  abnormal  hy- 
draulic indications. 

(2)  Detection  of  Existing  Casualties  - If  depth  change  is  ordered,  DO 
and  planesmen  note  that  the  plane  angle  indicators  do  not  respond 
or  are  erratic.  This  is  followed  by  feel  and  indicated  attitude  and 
depth  changes;  lag  varies  from  0.  5 sec  to  5.  0 sec.  Other  indica- 
tions may  be  a hydraulic  or  electric  alarm  or  an  audible  change  in 
the  noise  of  hydraulic  fluid  flow. 

If  a steady  state  is  desired,  planesmen  note  that  the  angle  indica- 
tor is  not  consistent  with  control  position.  Both  the  DO  and  planes- 
men note  erratic  indications  or  hydraulic  or  electric  alarm.  Both 
subsequently  note  "feel"  and  indicated  attitude  changes;  lag  varies 
from  0.  5 sec  to  5.  0 sec.  SP  may  have  difficulty  in  reaching  and 
holding  ordered  attitude. 

c.  Verification 

The  DO,  supported  by  detailed  H and  BCPO  checks,  verifies  that 
a fairwater  plane  casualty  exists.  DO  immediately  checks  emer- 
gency fairwater  plane  indicator  and  assesses  trim  and  maneuver 
effects.  Checks  by  DO  involve  short  delays,  checking  hydraulic 
control  in  the  emergency  mode,  and  checking  changes  in  ship’s 
speed  and  attitude. 

2.  DECISION  MAKING  (DO/OOD) 

a.  Ship  Characteristics  - Rate  of  change  and  current  values  of  speed, 
depth,  attitude,  and  turn  and  plane  angle  are  noted. 

b.  Crew  Characteristics  - Crew  capability,  state  of  training,  physio - 
logical/psychological  condition,  and  reaction  time  are  assessed. 

c.  System  Status  - Status  of  essential  ship  systems  is  assessed.  DO 
assesses  stern  planes  hydraulics,  and  MBT  blow.  OOD  assesses 


NAVTRADEVCEN  69-C -0298-1 


TABLE  2.  SEQUENCE  AND  TASK  ANALYSIS  (Continued) 


stern  planes,  hydraulics,  MBT  blow,  rudder,  propulsion,  engine 
order  telegraph,  1 MC  and  7 MC,  and  sound-powered  phone. 

d.  Recall  Appropriate  Data  - Special  instructions,  ship's  standing 
orders,  and  other  constraints,  are  recalled,  for  example,  mini- 
mum depth,  surface -ship  activity,  and  ice  overhead. 

e.  Recall  Available  Courses  of  Action  - The  available  courses  of 
action  are  stern  planes  rise,  fislhtail  rudder,  emergency  back, 
MBT  blow,  emergency  control,  'manual  control,  correct  fair- 
water,  plane  problem,  and  none. 

f.  Data  Collation  - The  information  of  2 ^a  through  2d^  are  collated 
against  alternatives  of  2e. 

g.  Course  of  Action  Selection  - Course  of  action  is  selected  from  2f . 

h.  Command  Decision  - DO  receives  direction  from  OOD  and  modi- 
fies or  retains  decision  of  2g. 

3.  CORRECTIVE  ACTION 

a.  Basic  Sequence  - DO  orders  stern  planes  positioned  to  minimize 
depth  excursion  and  OOD  notifies  engine  room  of  failure. 

b.  Representative  Alternative  Actions  - Depending  on  depth,  tactical 
situation,  and  special  instructions  the  DO/ OOD  may  elect  to  avoid 
noise  or  broaching  by  some  combination  of  ( 1)  delay  blow  MBT? 

(2)  speed  all  stop,  then  back  full;  (3)  fishtail  rudder;  and  (4)  flood 
negative . 

If  fairwater  plane  fails  at  a small  angle,  at  moderate  or  shallow 
depth  and  low  ship  angle,  DO/OOD  may  elect  to  attempt  to  free 
the  fairwater  planes  before  employing  the  stern  planes. 

If  hydraulic  failure  is  indicated  and  ship  conditions  permit,  DO/ 
OOD  will  elect  to  have  planesmen  switch  hydraulics  to  emergency 
or  the  manual  mode  as  the  primary  initial  action. 

If  indicator  failure  only,  DO/OOD  will  elect  to  have  planesmen 
use  emergency  indicator  only.  The  DO  may  have  phones  manned 
in  engine  room  to  monitor  fairwater  plane  ram  indicator. 
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TABLE  2.  SEQUENCE  AND  TASK  ANALYSIS  (Continued) 


4.  INFORMATION  FLOW  DIAGRAM 

detection  verification 


DECISION  ACTION 


5.  FOLLOW  ON  ACTION/FEEDBACK 

a. .  Results  - Indication  of  Results  of  Initial  Action 

(1)  Operator  Response  - Personnel  who  will  indicate  satisfactory  exe- 
cution of  orders  and/or  status  of  systems  for  which  they  are  re- 
sponsible are  the  DO,  EOOW,  H.  SP,  and  BCPO/COW. 

(2)  CO  Response  - The  CO  may,  at  his  option,  elect  to  supplement  or 
modify  the  actions  of  the  DO  or  OOD. 

(3)  Ship  Response 

(a)  Variables  of  2a  and  2c. 

(b)  Variable  and  negative  tank  status 

b.  Determine  Supplemental  Actions  Required  - Possibilities  include 
change  from  limited  action  of  3b  to  full  action  of  3a. 
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TABLE  2.  SEQUENCE  AND  TASK  ANALYSIS  (Continued) 


c.  Indication  of  Followon  Results 

(1)  CO  Response  (Directives)  - The  CO  may,  at  his  option,  elect  to 
supplement  or  modify  the  actions  of  the  DO/ OOD. 

(2)  Ship  Status  - Followon  conditions  of  recovery  procedure  are  slight 
change  in  buoyancy,  decreasing  depth,  decreasing  down  angle, 
speed  — 5 knots,  and  depth  = 400  ft  (assuming  ascent  from  deeper 
depths).  Followon. conditions  of  alternate  recovery  procedures 
are  positive  buoyancy,  decreasing  depth,  decreasing  speed,  and 
decreasing  down  angle. 

Determine  Action  Required  to  Complete  Recovery  - Desired  (safe) 
operating  envelope  and  action  required  are  identified. 

Complete  Casualty  Recovery 

Representative  action  for  completion  of  recovery  procedures  is  to 
adjust  stern  planes  and  trim  to  level  off  at  desired  depth  consistent 
with  tactical  situation. 

Representative  action  for  completion  of  alternate  recovery  pro- 
cedures are 

1.  Secure  blow  and  adjust  trim  to  level  off  at 
a desired  depth  consistent  with  tactical 
situation 

2.  Engines,  all  stop 

3.  Rudder  amidship 

4.  Blow  and  vent  negative  (or  hovering  tanks). 

Final  action  is  to  rig  for  ship  operation  within  a safe  operating  en- 
velope. 


(from  Goodyear , 1966) 
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TABLE  3.  CRITICAL  FACTORS  AND  SKILLS/ KNOWLEDGE  REQUIREMENTS  (Continued) 
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*No  direct  correlation  exists  between  numbers  assigned  to  items  in  middle  and  right-hand  columns, 
(from  Goodyear,  1966) 


TABLE  4.  VISUAL  AND  MOTION  CUE  REQUIREMENTS: 
INDIVIDUAL  TACTICAL  OPERATIONS 


1 

1 
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TABLE  4.  VISUAL  AND  MOTION  CUE  REQUIREMENTS: 

INDIVIDUAL  TACTICAL  OPERATIONS  (Continued) 
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TABLE  4.  VISUAL  AND  MOTION  CUE  REQUIREMENTS: 

INDIVIDUAL  TACTICAL  OPERATIONS  (Continued) 
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which  is  based  on  Guilford's  (1967)  structure-of-intellect  (SI)  model 
(Figure  10).  In  SI  theory,  human  intellective  acts  involve  a.n  operation, 
a content  and  a product.  These  three  dimensions  and  their  subgroupings 
permit  120  combinations  (triads).  What  Siegel  accomplished  was  to  iden- 
tify the  specific  tasks  performed  by  the  tactical  coordinator,  then  fit  these 
into  the  SI  framework,  and  estimate  task  difficulty  on  a scale  between  0 and 
100.  Specifically,  the  extent  to  which  each  of  the  14  SI  mental  functions 
(5  operations,  3 contents  and  6 products)  is  involved  in  each  of  the  50  tacti- 
cal coordinator  tasks  i was  estimated  on  the  100  point  scale. 

Schematically,  the  procedure  is  as  follows: 

develop  job  task  , ■ fit  the  tasks  into  ^ Compute  maxi- 
list (interaction  the  SI  model  ^ mum  involvement- 

with  subject  by -taxation 

matter  experts)  integrals  (maxi- 

mal intellectual 
triads) 

From  this  analytic  base  of  task  information,  judgments  were  made  on  the 
kinds  of  equipment  capabilities  needed  for  training  in  ASW  tactical 
problems . 


An  intellectual  load  analysis  (presented  in  Siegel  and 
Federman,  1968)  indicated  two  types  of  information:  which  aspects  of  the 

tactical  coordinator  job  imposed  the  heaviest  intellectual  load  (training 
requirements);  and  what  specific  kinds  of  intellectual  activities  were  in- 
volved when  the  man  was  heavily  loaded  (training  content).  For  this  speci- 
fic ASW  system,  computed  maximum  "involvement-by-taxation"  integrals 
(maximal  intellectual  triads)  indicated  a principal  requirement  for  training 
in  interpretive,  judgmental,  problem-solving  aspects  of  the  man's  job. 

The  so-called  executive  functions  of  the  tactical  coordinator,  the  "thinking" 
aspects  of  his  job  (i.e.,  analyzing  tactical  information,  determining -perti- 
nent information,  planning  courses  of  action,  making  decisions)  fell  out  as 
the  prime  training  requirement. 

2.3.  1.1.5  Interrelationship  Graphing --Another,  rather  unique  form  of 
rational  analysis  is  the  Interrelationship  Graph  (IRG)  technique,  developed 
specifically  for  computer  model  development  (Frye,  1965).  It  provides  a 
topological  representation  of  the  structure  of  a simulation  model  which 
identifies  all  model  elements  and  shows  relationships  between  these  ele- 
ments. The  IRG  is  a network-type  diagram  which  is  similar  in  definition 
to  other  forms  of  graphic  analysis  in  that  interconnected  geometrical 
figures  are  used,  but  they  represent  numbers  which  represent  quantities 
of  interest.  Considerable  symbology  is  utilized  in  the  IRG  which  cannot  be 
efficiently  described  here,  but  the  IRG  rules  are  simple  and  easy  to  learn. 
They  are  explicated  in  Frye  (1965).  The  primary  purpose  of  the  IRG  is  to 
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Figure  10.  Guilford's  Structure -of-Intellect  Model 
(from  Siegel  and  Federman,  1970) 
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serve  as  a communications  tool  for  those  working  with  digital  computer 
simulation.  The  technique  has  not  yet  been  fully  developed  and  refinements 
in  the  method  are  needed  as  well  as  explorations  into  its  use  for  other  pur- 
poses. For  example,  it  appears  useful  in  quantitative  analysis.  Sensitivity 
analysis  (perhaps  on-line  with  a computer)  appears  as  a promising  applica- 
tion. The  use  of  IRG's  to  define  sequences  of  training  which  are  to  be 
automated  (mission  modeling)  is  of  particular  interest  for  the  design  and 
utilization  of  synthetic  training  systems. 

•* 

2.3.  1.2  Summary  of  Task  Analysis  Methods.  Task  analyses  methods 
have  the  feature  in  common  that  all  are  rational  approaches  which  attempt 
to  categorize  system  performances  for  the  purpose  (i.e.  , our  interest)  of 
training  system  design.  They  are,  at  best,  heuristic  descriptions  of  acti- 
vities at  the  functional  interface  of  the  man  and  that  with  which  he  interacts. 
Today,  task  analysis  methods  leave  much  to  be  desired  for  training  system 
design.  The  methods  continue  to  be  cumbersome,  ill-defined  and  non- 
standard, what  Miller  (1962)  has  called  the  "limping  practices  of  the 
present.  " 

Experience  with  task  analysis  has  made  clear  the  fact  that  its 
content  is  not  generalizable  across  training  situations — it  is  tailored  to  the 
needs  of  the  system  contemplated.  Unfortunately,  the  process  of  deter- 
mining the  task  requirements  is  complex  and  difficult  and  the  available 
procedures  are  inexact  and  cannot  be  applied  in  any  standard  way.  The 
process  is  subjective  and  the  adequacy  of  the  results  is  due  in  large  part 
to  the  skill  of  the  analyst.  At  present,  the  form  and  content  of  the  informa- 
tion derived  from  task  analysis  is  such  that  it  is  not  readily  translatable 
into  design  specifications  . 

2.3.  1.2. 1 Research  Issues.  A number  of  issues  can  be  identified  all 

of  which  can  be  gathered  under  a single  purpose- -developing  the  form  and 
content  of  task  information  so  that  the  output  is  directly  useful  in  decisions 
on  training  device  design. 


An  issue  of  some  consequence  is  the  definition  of  what  is 
expected  (and  also  not  expected)  in  task  information  for  device  design  de- 
cisions . This  relates  directly  to  the  current  problem  of  the  inability  to 
translate  task  information  directly  into  design  terms.  Thus,  an  effort 
should  be  devoted  to  the  development  of  criteria  to  determine  what  is  of 
consequence  in  defining  tasks  to  be  trained.  Refinements  are  needed  be- 
yond the  current  practice  of  considering  tasks  which  are:  important  to 

system  effectiveness;  require  high  skill  requirements;  and  will  yield  im- 
proved performance  through  training.  We  suspect  that  this  development 
must  occur  around  families  or  classes  of  training  systems  because  of 
significant  differences  in  task  requirements,  specifically  among  (1)  OFT's 
involving  motion  and  outside -visual  world  requirements;  (2)  tactical  team 
trainers  emphasizing  tactical  decision  making;  and  (3)  generalized  trainers 
emphasizing  major  functions  to  be  trained. 
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Detailed  task  classification  data  are  also  required  but  far  in 
advance  of  the  rudimentary  task  taxonomies  currently  on  the  scene.  These 
data  should  be  organized  so  that  eventually  common  task  factors  pertinent 
to  classes  of  training  systems  will  become  available  to  designers  to  be  used 
in  checklist  or  dictionary  fashion,  thus  obviating  the  time,  expense  and 
effort  required  to  analyze  each  new  situation  from  the  ground  up.  The  data, 
once  organized  and  available,  preferably  in  a computerized  data  bank,  would 
be  pertinent  to  any  device  design  situation  and  thus,  would  cut  drastically 
the  time  for  any  new  analytic  effort. 

Current  behavioral  taxonomies  are  of  limited  value  in  devel- 
oping training  objectives  for  a device.  These  have  attempted,  with  limited 
success,  to  obtain  a set  of  behavioral  elements  or  categories  which  can  be 
grouped  into  defined  sets  of  distinct  task  categories  so  that  patterns  of  task 
attributes  can  be  prescribed  for  specific  training  situations.  What  is  needed 
is  a standard  set  of  terms  which  can  be  used  directly  in  describing  opera- 
tional activities. 

Allied  with  the  above  is  the  need  for  a "short  cut"  task  anal- 
ysis technique  or  families  of  techniques  which  have  the  power  needed  for 
determining  training  requirements,  but  tailored  to  the  needs  of  the  system 
under  study.  This  would  be  of  considerable  value  in  developing  the  func- 
tional requirements  for  any  contemplated  training  system,  for  time  con- 
straints and  the  complexity  of  the  systems  quite,  often  preclude  the  organi- 
zation of  task  data  to  the  level  required  for  effective  human  factors  design 
inputs  . ' 

2.3.2  Training  Objectives.  The  effectiveness  of  the  training  system 
design,  i.e.  , how  the  training  environment  is  provided  to  facilitate  the 
necessary  learning,  depends  also  on  a clear  understanding  and  expression 
of  the  objectives  of  the  training  to  be  accomplished.  Explicit  statements  of 
training  objectives  are  needed  which  define  training  content.  A relevant, 
complete  and  well  organized  list  of  these  training  objectives  serves  as  an 
important  guide  for  decisions  on  what  must  be  simulated  (extent  and  preci- 
sion of  the  simulation  parameter s)  . The  desired  form  for  specifying  train- 
ing objectives  is  we  11 -documented.  So  far  as  their  structure  for  training 
device  design  is  concerned,  a training  objective  must  specify  what  is  to  be 
trained  with  a description  of  what  the  individual  or  the  team  must  be  able 
to  accomplish  at  the  end  of  training  to  satisfy  the  objective.  Included  is  a 
statement  of  the  conditions  under  which  mastery  of  the  objective  is  demon- 
strated, and  where  applicable,  the  standard  of  performance  (minimum 
level  acceptable)  describing  the  attainment  of  the  objective.  Each  objective 
is  stated  in  terms  of  observable  performance  so  as  to  permit  the  measure- 
ment of  its  attainment.  Thus  , for  each  objective , the  following  is  required: 

• Performance  elements  ' These  are  achieved  from  the 

definition  of  task  structure . The 
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task  analysis  yields  task  state- 
ments (skill  and  knowledge 
requirements)  which  serve  as 
the  performance  element  (i.e.  , 
what  is  going  to  be  trained)  . 

# Measurability  Measuring  and  scoring  of  per- 

formance is  tailored  to  the  training 
objectives  (see  Phase  5B  of  this 
section)  . 

• Conditions  of  This  specification  is  obtained 

performance  from  the  analysis  of  the  opera- 

tional system  (see  Phase  2 of 
thi  s sec  tion)  . 


The  development  of  training  objectives  is  based  on  the  task  struc- 
ture, with  task  statements  serving  as  the  performance  element.  Our 
interest  is  in  terminal  objectives,  these  define  meaningful  units  of  perform- 
ance, relevant  to  real-world  operations,  which,  when  organized,  describe 
the  requirements  for  training.  They  serve  as  the  framework  for  organizing 
the  training  system  and  for  developing  the  system  design.  This  component 
skills  and  knowledges  the  trainee  must  learn  are  anchored  to  these  terminal 
objectives  . 


2. 3.2.1  Sequence  of  Steps  in  Developing  Training  Objectives.  The  pro- 
cess of  developing  job  relevant  training  objectives  pertinent  to  device  design 
decisions  is  outlined  in  the  following  progression: 


Step  1:  Describe  and  analyze  the  operational  system — 

Pertinent  information  on  the  operational  system 
is  assembled  which  includes  the  description  of  the 
system  and  its  components,  the  missions,  the 
operational  and  tactical  environment,  and  the 
primary  mission  profiles  (accomplished  during 
Phase  2)  . 


Step  2:  Define  task  structure  - -An  inventory  is  made  of 

the  duties  associated  with  each  position  and  the 
tasks  associated  with  each  duty. 


Step  3:  Conduct  task  analyses- — Formal  task  analyses 

are  conducted  to  determine  detailed  task  struc- 
tures and  to  assess  the  importance  and  criticality 
of  tasks  for  training,  e.g.,  tasks  that  are  important 
to  system  effectiveness  , have  high  skill  require- 
ments , and  will  yield  improved  performance  through 
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training.  No  well  developed  guidelines  can  be  given 
for  defining  the  criteria  to  determine  what  to  train. 

Most  often  the  decision  rules  place  most  weight  on 
judgments  reflecting  on  the  rationale  given  above  and 
crude  distinctions  are  made  (e.g.  , a level  of  1 may 
mean  that  the  task  should  not  be  taught  at  all  since  it 
is  not  judged  important  to  mission  success,  or  can  be 
easily  acquired  on  the  job;  a level  of  3 rnay  mean  that 
the  task  is  important  to  mission  success  and  is  difficult 
to  teach  on  the  job,  has  high  skill  requirements,  and 
so  on)  (see  Section  2.3.1). 

Step  4:  Prepare  detailed  task  de  scriptions  - -For  those  tasks 

that  have  been  identified  for  the  training  system,  de- 
tailed task  descriptions  are  prepared.  The  goal  of 
this  effort  is  to  determine  the  skills  and  the  required 
knowledges  involved  in  task  performance.  The  result 
is  an  organization  of  tasks  to  be  performed  and  the 
component  skills  and  knowledges  that  are  involved  in 
each  task.  The  task  statement  defines  the  performance 
elements  in  the  expression  of  the  training  objective. 

Step  5:  Express  the  task  statements  in  the  form  of  training 

objectives- -When  the  intended  performance  situation 
is  defined,  the  meaningful  units  of  performance  are 
identified  and  selected  and  the  necessary  immediate 
learning  needs  are  established.  These  should  describe 
clearly  what  the  trainee  must  do  to  satisfy  the  objective. 
As  indicated  earlier,  the  performance  element  of  the 
objective  is  derived  from  the  task  structure  (intended 
performance  situation)  . 

In  addition,  the  conditions  under  which  performance  is 
to  be  attained  form  an  important  part  of  the  objective 
since  a number  of  factors  influence  the  difficulty  level 
of  training  as  a function  of  task  loading.  The  conditions 
to  account  for  can  be  categorized  as  follows: 

• physical  environment — involved  here  are  environ- 

mental effects  such  as  visual,  aural,  motion,  and 
earth  atmosphere  (sea  conditions,  conditions  of 
flight,  and  underwater  conditions). 

. task  loading- -involved  here  are  issues  of  perform- 
ance complexity  arising  from  increasing  task 
demands  and  the  tactical  environment  which  define 
the  amount  of  "problem  clutter"  in  generating  the 
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situation  for  training.  The  factors  subsumed  under 
tactical  environment  include:  target  configurations, 

speed  of  an  engagement,  communications  require- 
ments, friendly  force  composition  and  disposition, 
and  enemy  threat. 

• mission  complexity — task  requirements  are  differ- 
ent  depending  on  the  type  of  mission  to  be  accom- 
plished or  on  that  portion  or  segment  of  the  mission 
of  primary  concern.  For  example,  in  Air  Force 
combat/ logistics  flight  missions,  considerable 
differences  in  task  sets  are  found  in  the  primary 
delivery  segment  of  the  missions,  which  call  for 
quite  different  training  requirements  (Smode,  Post 
and  Meyer,  1966;  Smode,  Hall  and  Meyer,  1966). 
Also,  the  geometry  of  a simulated  engagement  is 

of  prime  significance  in  defining  the  conditions 
underlying  a training  objective. 

• target  maneuvering — tactical  situations  vary  signi- 
ficantly as  a function  of  target  activity  (evading/ 
attacking,  maneuvering,  such  as  altitude  changes 
in  aircraft  or  submarine  movements  above  and 
beneath  the  layer)  and  the  relative  geometry  be- 
tween own- system  and  a target(s). 

• non-normal  operations — malfunctions  and  emer- 
gency situations  require  specification  (as  pertinent) 
in  the  training  objective. 

The  final  component  in  the  training  objective  is  the  specification 
of  the  standard  of  performance  (as  pertinent)  that  must  be  met  to  indicate 
achievement  of  the  objective.  This  maybe  in  such  terms  as  error  toler- 
ance, speed  of  accomplishment,  or  procedural  accuracy. 

Ammerman  and  Melching  (1966)  ha.ve  reported  their  work  on 
the  development  of  training  objectives  (which  is  part  of  the  systematic 
effort  by  HumRRO  on  the  design  of  training  systems).  Although  their  work 
has  centered  more  broadly  on  educational  contexts  and  on.  training  situa- 
tions in  Army  Service  Schools  (curriculum  development),  the  material  is 
nevertheless  of  value  for  the  design  of  synthetic  training  systems  . ^ 

^For  additional  information,  see  Mager  (1961),  Tyler  (1964),  Smith  (1964), 
and  Rundquist  (1967,  1970). 
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A distinction  i_  made  by  these  writers  between  terminal  objectives  and  en- 
abling objectives.  Terminal  objectives  define  meaningful  units  of  perform- 
ance that  are  relevant  to  real-world  requirements  and  are  critical  for 
instruction.  They  denote  final  criterion  performance  and  are  valid  repre- 
sentations of  the  instructional  goEils  . Terminal  objectives  serve  as  the 
framework  for  organizing  the  training  system  and  as  the  anchor  for  identi- 
fying the  enabling  skills  and  knowledges.  They  provide  the  base  for  devel- 
oping the  system  design.  Enabling  objectives  are  not  instructional  goals 
per  se  but  represent  immediate  learning  demands  for  attaining  the  terminal 
objectives.  These  define  what  the  trainee  needs  to  learn  and  consist  of  the 
component  knowledges  and  skills  that  must  be  mastered  in  order  to  achieve 
the  terminal  objectives.  Thus,  they  bridge  the  gap  between  entry-level 
capability  and  end  of  instruction  goals.  Since  they  represent  the  learning 
difference  between  where  the  trainee  is  now  and  where  he  should  be,  they 
are  derived  from  the  knowledge  of  the  trainee  population  characteristics 
and  from  the  terminal  objectives  and  they  describe  the  means  to  attain  the 
directly  meaningful  terminal  objectives.  The  sequence  of  development  in 
the  design  for  training  proceeds  as  follows: 


Terminal  objectives  can  vary  in  a number  of  ways  and  thus, 
their  utility  and  communicability  are  influenced.  Ammerman  and  Melching 
(1966)  have  classified  terminal  objectives  in. terms  of  five  factors  which 
account  for  their  differences. 
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• Type  of  performance  unit- -the  objective  may  refer  to  three 
types  of  meaningful  units  of  performance.  These  are: 
specific  tasks  (e.g.  , performance  gyro  azimuth  transmission 
check  on  a Nike  Hercules  radar);  generalized  skills  (e.g.  , 
adjust  the  carburetor  of  any  type  of  gasoline  engine  used  on 
land  vehicles);  and  generalized  behaviors  (e.g.  , maintain 

an  awareness  of  safety  hazards  when  working  in  a machine 
shop) . 

• Extent  of  action  description- -objectives  may  show  consider- 
able variation  in  the  level  of  description.  This  may  range 
from  a statement  of  action  only,  to  descriptions  of  repre- 
sentative or  complete  actions  that  are  indication's  of  the 
desired  behavior. 

• Relevancy  of  trainee  action- -this  refers  to  the  relatedness 
of  the  behavior  of  the  trainee  to  that  required  in  the  job 
situation.  Objectives  tend  to  be  highly  relevant  when  they 
are  derived  from  an  analysis  of  the  job  situation  (e.g.  , field 
strip  a heavy  machine  gun). 

• Completeness  of  structural  components  — statements  of  ob- 
jectives are  complete  to  the  extent  the  components  (action, 
condition,  standard)  are  included. 

• Precision  of  each  structural  component — this  factor  deals 
with  the  explicitness  of  each  component  in  the  objective. 

2. 3. 2.2  A Sampling  of  Training  Objectives.  Several  examples  of  train- 
ing objectives  are  shown  below. 

Example  1:  This  is  a simple  example  of  a terminal  training 

objective  for  individual  operator  task  performance  (Ammerman 

and  Melching,  1966). 

• The  trainee,  using  rubber  and  friction  tape,  pliers 
(TL-13-A)  and  wire  (WD-l/TT),  should  be  able  to  make 
a standard  field  wire  splice  by  completing  each  of  the 
following  steps  in  sequence. 

From  one  conductor,  cut  off  one  plier's  length 
(about  six  inches). 

- Mark  each  conductor  six  inches  from  end  by 
inserting  one  conductor  at  a time  into  small 
hole  of  jaws  of  pliers. 

Close  pliers.  s '; 
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Insert  long  conductor  in  small  hole  about 
two  inches  from  end. 

Close  jaws  carefully,  remove  insulation. 

Repeat  procedures  for  each  conductor. 

Tie  long  conductor  of  one  pair  to  short 
conductor  of  second  pair,  using  a square 
knot . 

Etc . 

Additional  examples  of  training  objectives  for  individual  opera- 
tor tasks  are  shown  below  (Ammerman  and  Melching,  1966;  Rundquist, 
1967). 


• The  trainee  should  be  able  to  field  strip  the  major 
components  of  an  M-  14  rifle  under  conditions  of  total 
darkness  within  five  minutes. 

• The  trainee  must  detect  errors  in  acquiring,  processing, 
displaying  and  reporting  of  contact  data  during  a pro- 
gramed exercise  consisting  of  representative  samples 

of  audio-taped  radar  operator  reports  and  corresponding 
pictorial  radar  presentations.  Task:  "monitors  surface 

search  radar  operator  in  search  for  and  detection  of  surface 
contacts,  and  processing,  display  and  reporting  of  contact 
data."  (Since  in  this  instance,  the  performance  to  be  taught 
with  taped  reports  and  pictures  of  radar  scopes  is  different 
from  the  task  on-the-job,  the  task  itself  is  given  for 
reference)  . 

Example  2:  Single  Destroyer,  Single  Submarine  Tracking 

and  ASROC  Attack  (reference:  Section  IV,  this  report,  ASW 

operations  in  device  14A2) . 

• Conduct  basic  ASROC  attacks,  achieve  three  complete 
fire  control  solutions  and  demonstrate  weapon  launch 
procedural  accuracy  in  all  subteam  performances. 

Team  must  satisfactorily  complete  fire  control  solu- 
tion and  ASROC  (Mk46/1)  launch  procedures  in -attack- 
ing a maneuvering  submarine  at  various  ranges  and 
courses  relative  to  own-ship.  Secondary  emphasis  is 
placed  on  accuracy  and  speed  of  performance. 


O 
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Example  3:  SH-3  Helicopter  46/1  Torpedo  VECTAC  from 

Sector  Screen  (reference:  Section  IV  this  report,  ASW 

operations  in  device  14A2)  . 

• Implement  a coordinated  search  and  attack  on  a 

conventional  submarine  attacking  the  screen.  Positive 
control  of  the  helicopter  will  be  effected  and  the  follow- 
ing accomplished: 


tfeadwind  vectoring  procedures  while  positioning 
Helo  in  search  sector. 

- Positioning  the  Helo  in  square  search  pattern  of 

operation  PUMPKIN  (51S)  using  crosswind  vectoring 
technique. 


Implementing  an  urgent  SH-3  46/1  VECTAC. 

Full  reporting  procedures  over  RT  Nets  using  the  scramble 
Table  will  be  accomplished. 

Example  4:  Training  objectives  developed  for  a Submarine 

Casualty  Control  Trainer  (Goodyear,  1966). 


Seven  functional  training  units  (i.e.  , closely  related  and  inte- 
grated subject  matter  areas  for  training)  were  identified.  • 


1.  Ship  Command  and  Control 

2.  Fire  recognition/isolation/reporting 

3.  Flooding  recognition/isolation/ reporting 

4.  Atmospheric  contamination  recognition/ isolation/reporting 

5.  Propulsion  casualties  ship  control  effects 

6.  Electrical  casualty  recognition/isolation/reporting 

7 . Ship- systems  monitoring  and  control* 

Within  each  of  these  7 functional  training  units,  a series  of 
training  objectives  were  prepared.  The  example  given  is  for  only  one,  - 
the  Command  and  Control  Casualty  training  unit.  The  following  instruc- 
tional aims  were  defined.. 


Ensure  a high  degree  of  tracking  capability  by  plane smen 
so  that  cues  of  a malfunction  for  indicators,  control  system 
or  ship  feel  are  not  confounded  with  erratic  planes  or  depth 
control  (high  fidelity  control  capability). 

Develop  in  plane  smen  proper  scanning  habits  among  indi- 
cators (physical  representation  of  diving  panel  and  BCP 
display  arrangements  identical  to  the  class  of  ship) . 
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. Develop  discriminating  responses  to  specific  cues  of  normal/ 
abnormal  performance  (depth,  speed,  and  cros s -coupling 
effects  on  ship  accelerations  in  the  X-Z  plane  should  be 
highly  similar  to  the  effects  of  plane  and  flooding  casualties 
and  casualty  recovery  actions)* 


• Develop  ability  to  shift  to  alternate  modes  of  operation, 
such  as  rate  control,  manual  overrides  for  emergency 
blow  or  plane  control  and  local  manual  control  of  planes, 
rudders  and  vents  (high  fidelity  simulation). 

. Ability  to  analyze  and  maintain  trim  control  so  that  cues 
of  a malfunction  from  indicators  , control  system,  or  ship 
feel  are  not  confounded  with  a heavy  condition,  down  angle, 
up  angle,  or  list  due  to  improper  trim  (high  fidelity  trim 
analysis  and  control). 

, Ability  to  recognize,  diagnose  and  correct  for  depth  and 
depth-rate  control  problems  due  to  indications  of  stern 
plane,  fairwater  planes,  or  rudder  casualties. 


# Ability  to  initiate  ship  control  actions  in  response  to  gen- 
eral emergencies  and  alarms.  This  includes  emergency 
surfacing,  depth  change  for  emergency  ventilation  and 
depth  change  to  ordered  depth. 


. Develop  in  the  team  the  skills  in  evaluating  the  progressive 
effects  of  recovery  action  and  performance  ox  follow-up 
action  necessary  to  restore  a safe  operating  envelope. 

(High  fidelity  ship  performance  and  ship  systems  response; 
should  be  equipped  to  reveal  depth  excursion,  air-bank 
management,  including  negative  recoverability  effects.) 

• Exercise  the  ship-control  team  in  communications  and 
team  interactions.  Emphasis  is  placed  on  coordinating 
control  of  depth  and  trim  angle . 


2. 3.2.3  Organizing  the  Training  Objectives  . Since  a number  of  train- 
ing objectives  will  be  identified  for  any  given  training  system  it  is  neces- 
sary to  organize  them  so  that  the  full  range  of  device  capabilities  needed 
can  be  identified.  The  objectives/  are  usually  graded  in  difficulty  in  terms 
of  the  factors  identified  above,  vis-a-vis  the  task  content.  This  grouping 
can  be  accomplished  via  two  dimensions. 


'The  first  is  the  procedural  to  the  fully  integrated  (or  tactical) 
dimension  whereby  the  early  or  initial  objectives  are  concerned  with  pro- 
cedural adequacy  , e . g.  , the  ability  to  operate  basic  equipments  , knowledge 
of  the  activity  sequences  , plotting,  communicating , etc  . From  this  are 
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built  up  more  complex  objectives  (in  which  the  former  are  included  in  that 
they  represent  building  blocks  for  more  complex  activities).  Thus,  a later 
set  of  objectives  would  emphasize  full  utilization  of  those  portions  of  the 
system  to  be  simulated,  such  ac  , the  tactical  employment  of  a system 
against  an  enemy  (e.g.  , general  purpose  training  in  tactical  ECM,  involv- 
ing defensive  ECCM,  Jamming,  F.adar  Homing  and  Warning,  and  Recon- 
naissance and  ELINT,  such  as  provided  by  device  15E18). 


A second  dimension  specific  to  team  trainers  is  the  organiza- 
tion of  objectives  at  individual/ subteam  levels  and  progressing  to  team 
sequences  involving  the  range  of  complexities  occurring  in  real-world 
operations  . A good  example  of  this  is  provided  in  Section  IV  for  ASW 
operations  in  device  14A2 . Three  groupings  of  objectives  were  developed 
to  provide  a series  of  training  exercises  graduated  in  difficulty.  The  first 
block  centered  on  basic  attack  procedures  (non-mission  context)  in  which 
the  objectives  specified  the  development  of  fire  control  solutions  and  launch 
procedures  for  the  relevant  weapons.  The  next  block  of  objectives  was 
devoted  to  single  ship  sequences  in  the  screen  mission  context  where  the 
previous  block  requirements  were  imbedded  in  a total  mission  from  search 
and  detection  through  weapon  assignment  and  target  prosecution.  The  final 
block  dealt  with  multi-unit  missions  (Search  and  Attack  Unit  (SAU)  context) 
involving  more  complex  tasks  of  coordinating  the  operations  of  other  units. 
This  represents  the  fullest  expression  of  coordinated  team  training  objec- 
tives. Thus,  a graduated  series  of  training  exercises  are  developed  and 
the  strategy  for  training  is  based  on  an  interrelated  series  of  specific 
training  objectives. 

2. 3. 2.4  Summary  of  Research  on  Training  Objectives.  Simply  speaking, 
the  training  objective  is  a precise  means  of  communicating  instructional 
goals.  It  is  in  a form  most  useful  for  device  design  when  it  is  stated  ex- 
plicitly in  terms  of  performances  expected  of  trainees  at  the  end  of  instruc- 
tion, rather  than  in  terms  of  training  content  or  subject  matter. 


The  present  development  of  the  training  objective  has  emerged 
from  the  confluence  of  work  in  education  (see  Tyler,  1964),  the  work  in 
programed  instruction  (see  Mager,  1961)  and  the  sizeable  effort  in  human 
factors  and  training  research  on  the  development  of  rational  analysis  tech- 
niques for  describing  task  structure  in  complex  man- machine  systems. 
While  significant  and  meaningful  development  of  training  objectives  has 
been  accomplished  in  the  decade  of  the  1960*s,  the  question  of  meaning  is 
this:  have  behaviorally  stated  objectives  had  a significant  impact  on  the 

design  of  instructional  systems?  Melching  (Melching,  et  al,  1966)  also 
articulates  several  key  issues:  are  objectives  prepared  on  the  same  sys- 

tem comparable  among  writers,?  have  existing  statements  of  objectives 
satisfied  the  criteria  of,  the  performance  element,  the  conditions  under 
which  competence  is  demonstrated,  and  the  standard  of  performance? 

Has  the  nature  of  the  behavior  of  the  trainee  been  clearly  communicated? 
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The  answers  are  most  clearly,  no.  Training  objectives  that  have  been  pub- 
lished exhibit  no  standard  pattern.  They  vary  considerably  in  explicitness 
and  detail,  in  the  relationship  they  bear  with  the  performance  situation  and 
in  the  type  of  training  system  involved.  Most  statements  of  objectives  fail 
to  include  the  important  conditions  and  the  standards  of  performance  rele- 
vant to  the  training  action. 

Ammerman  (in  Melching,  et  al,  1966),  has  identified  several 
major  sources  of  variation  in  statements  of  training  objectives.  The  first 
is  in  the  level  of  specificity  with  which  the  trainee  action  is  stated.  As 
mentioned  earlier,  the  terminal  objective  is  associated  with  meaningful 
units  of  performance  and  it  establishes  the  criterion  performance  that  is 
to  be  attained  in  the  training  system.  'The  enabling  objectives  to  be  mean- 
ingful (i.e.  , achieve  organization),  are  anchored  to  the  terminal  objectives. 
It  is  the  specificity  of  the  enabling  objectives  that  is  cause  for  questioning 
their  usefulness  (i.e.  , each  consisting  of  one  component  action  or  knowl- 
edge to  be  learned  to  attain  the  terminal  objective). 

The  second  source  of  variation  is  the  extent  to  which  the  action 
is  described.  Most  examples  do  not  contain  any  descriptions  beyond  stating 
the  action  and  are  quite  nebulous.  Extensive  descriptions  of  trainee  actions 
are  required  to  provide  the  necessary  communicative  power.  For  example, 
the  additional  description  might  be  a listing  of  sequential  steps  for  a proce- 
dural task,  or  a listing  of  observable  actions  which  if  exhibited  would  be 
evidence  for  attainment  of  the  terminal  objective . 

A third  source  of  variation  is  the  completeness  of  the  state- 
ment of  the  objective.  This  refers  to  the  completeness  of  the  components 
of  a training  objective  (i.e.  , action,  conditions,  standard).  The  major 
neglect  is  in  the  depiction  of  the  conditions  of  performance  and  the  stand- 
ards of  performance. 

A fourth  source  of  variation  is  the  relevance  of  the  performance 
stated  in  the  objective.  This  is  the  validity  requirement  which  too  often  is 
not  met,  presumably  due  to  a lack  of  knowledge  about  the  exact  job  perform- 
ance requirements. 

Confusion  in  stating  training  objectives  has  resulted  from  an 
inability  to  distinguish  between  terminal  and  enabling  objectives 
(Ammerman  and  Melching,  1966).  Terminal  objectives  represent  end-of- 
training  capabilities  sought  by  the  instructional  system  and  they  are  deter- 
mined by  relevance  to  the  operational  situation  represented  in  training. 
Enabling  objectives  are  the  necessary  training  tasks  that  bridge  the  gap 
between  entry-level  trainee  capability  and  the  desired  outcome  as  defined 
by  the  terminal  objective.  Thus,  enabling  objectives  depend  on  terminal 
objectives  for  their  value;  both  are  essential  to  meaningful  training  design. 
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2. 3. 2. 5 Research  Issues.  The  importance  of  performanbe-oriented 
training  objectives  is  well  understood  in  training  requirements  analyses. 

Two  issues  are  prominent  for  research.  The  first  concerns  the  value  and 
impact  of  training  objectives  for  training  device  design.  The  major  emphasis 
on  performance  objectives  to  date  has  been  in  curriculum  design  and  test 
construction.  So  far  as  synthetic  training  systems  are  concerned,  their 
greatest  impact  is  felt  during  device  utilization  rather  than  in  the  design 
process.  Once  the  device  is  on-line,  training  objectives  are  central  in 
developing  an  organized  training  program  for  device  utilization  based  on  a 
series  of  training  exercises  graduated  in  complexity  and  difficulty.  What 
is  needed  is  research  to  define  procedures  for  incorporating  training  objec- 
tives in  the  process  of  human  factors  design  of  synthetic  training  systems. 

Aligned  with  the  above  is  the  research  need  to  standardize  the 
component  structure  of  training  objectives  for  use  in  device  design,  espec- 
ially for  classes  of  training  devices  having  similar  components  (sensor 
systems,  weapons  systems,  etc.).  Since  enabling  objectives  represent 
task  events  and  sequences,  this  should  tie  in  meaningfully  with  research  on 
task  commonalities.  The  work  on  task  taxonomies  relating  task  structure 
to  instructional  methods  is  relevant  here. 

2.3.3  Definition  of  Simulation  Elements.  The  reason  for  ‘he  operational 
job  analysis  is  to  achieve  an  idea  of  task  structure,  which  includes,  a de- 
scription of  the  trainable  critical  tasks  and  an  integration  of  these  tasks  by 
positions . The  range  of  these  task  requirements  to  be  achieved  by  training 
provide  the  clues  for  determining  what  is  to  be  simulated,  and  it  is  the 
"what  to  simulate"  (i.e.,  extent  of  simulation  in  the  trainee  compartment(s) 
in  functional  terms)  that  is  an  important  human  factors  contribution  to 
device  design. 


Thus,  we  are  ready  to  begin  a key  effort,  that  of  defining  the  en- 
vironment in  which  learning  will  take  place,  i.e.  , specifying  the  extent  of 
simulation  required  via  the  design  options  selected.  The  range  of  task  re- 
quirements provides  the  clues  for  the  numbers  and  types  of  parameters 
needed  in  simulation.  The  identification  of  simulation  elements , i.e.  , 
those  factors  to  be  simulated  that  determine  the  representativeness  and 
complexity  of  the  tasks  which  are  defined  for  training,  and  the  parameter 
values  (range,  envelope,  number)  are  the  crucial  human  factors  inputs  to 
the  definition  of  fidelity  of  simulation  in  the  trainee  compartment.  These 
simulation  elements  must  be  clearly  specified  in  order  to  achieve  the  re- 
quired device  configuration  to  enable  training  to  the  level  expected  (train- 
ing objectives)  , and  also  to  have  a flexibility  for  hand’ing  additional  train- 
ing objectives  should  they  emerge  after  on-line  experience  with  the  device. 

The  basic  human  factors  effort  is  to  identify  the  simulation  ele- 
ments to  be  represented  for  training,  for  these  define  the  device  hardware 
and  the  software  programing  requirement^ 'At  stake  are  the  following: 
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how  much  to  simulate  (tasks  to  be  trained),  and  what  levels  of  fidelity  to 
provide  (with  the  implicit  cost-effective  question  of,  what  can  be  omitted 
from  complete  fidelity  of  representation  without  significant  decrease  in 
training  value?).  The  simulation  engineer  needs  to  know  how  much  of  a 
task  to  represent,  namely,  what  and  how  many  parameters  must  be  simu- 
lated to  represent  the  task. 


Thus,  the  classes  of  simulation  elements  that  must  be  provided 
in  the  training  environment  (trainee  compartment)  to  achieve  the  defined 
and  the  anticipated  training  objectives  are  identified  at  this  time.  These 
classes  of  elements  refer  to:  own  vehicle  units;  targets;  and  media.  The 

matrix  shown  in  Figure  11  is  helpful  in  identifying  the  categories  of  factors 
relevant  to  the  system  under  consideration.  The  values  of  the  simulation 
elements  (number,  range,  envelope)  are  specified  in  some  detail  subse- 
quently in  Phase  5,  which  is  devoted  to  defining  the  characteristics  of  the 
operational  environment  to  simulate  for  training. 


In  the  present  phase,  attention  is  also  devoted  to  outlining  a use- 
able  measurement  system  to  provide  measures  and  scores  for  evaluating 
the  achievement  of  the  training  objectives.  This  effort  is  completed  sub- 
sequently in  Phase  SB  concerned  with  the  management  of  training  at  the 
instructor  station. 

2.3.4  Output.  The  completion  of  this  phase  yields  the  identification  of 

■ what  training  is  needed,  vis-a-vis  transfer  of  training.  .Task  structure  is 
put  into  perspective  wherein  task  statements  are  listed  depicting  the  acti- 
vities performed  by  duty  positions,  together  with  the  skill  and  knowledge 
requirements.  Training  objectives  and  the  measurement  requirements 
for  assessing  training  effectiveness  are  also  determined . Again,  it  is 
emphasized  that  the  reason  for  this  analysis  is  to  provide  a basis  for  deter- 
mining the  extent  of  the  simulation  required  in  the  trainee  compartment( s), 
culminating  in  the  identification  of  the.  relevant  simulation  elements,  to  be 
included  in  the  design  of  the  system  under  consideration 

The  activities  completed  thus  far,  now  enable  the  gross  definition 
of  the  training  system  (i.e.  , initial  correlation  between  that  piece  of  the 
operational  environment  to  -be  simulated  and  the  operational  system  in  its 
present  form)  . ■ ; : 

2.4  PHASE  4:  GROSS  DEVICE  DEFINITION.  The  definition  of  the 

boundaries  of  the  training  system  can  now  be  accomplished.  This  is  the 
gross  hardware  orientation,  centering  on  the  trainee  compartment  require- 
ments and  represent s : the  correlation  between  that  piece  of  the  operational 
universe  to  be  simulated  to  meet  the  user's  needs,  and  the  operational 
system  as  it  is  or  will  be  organized  and  employed. 
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Figure  11.  Simulation  Elements  Matrix 
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administrative  inputs  are  organized  until  the  functional  description  of  the 
training  system  is  completed. 


2.5  TRAINING  DEVICE  COSTS  VS.  TRAINING  VALUE.  The  specifica- 
tion of  design  for  a training  system  under  development  must  take  into  account 
the  issue  of  costs  vs.  training  benefits.  Tradeoff  analyses  are  required  to 
put  into  perspective  the  training  value  or  advantage  in  operational  capability 
afforded  by  a design  alternative  per  dollar  spent.  This  is  a difficult  thing 
to  do,  and  at  present  there  are  no  useful  methods  available  for  this  deter- 
mination that  most  people  can  agree  upon.  Most  often,  these  decisions  are 
made  intuitively  based  on  previous  experiences,  rarely  have  they  ever  been 
Mjust  right,  11  and  the  tendency  has  been  to  be  generous  in  engineering  features 
just  to  make  sure  that  the  user  has  what  he  believes  he  needs  to  accomplish 
training.  Often,  over-design  is  the  result,  with  no  clear  correlate  in  height- 
ened training  returns. 


Cost  tradeoffs  may  be  influenced  substantially  by  constraints  having 
no  technical  basis  . Foremost  among  these  are  features  such  as,  a limited 
dollar  budget  available  which  forces  an  undesirable  austerity  on  design;  and 
requirements  for  simulating  certain  ,,demandedf,  operational  characteristics 
which  unfortunately  contribute  minimally  to  training. 


The  cost/effectiveness  determination  is  a difficult  undertaking  since 
it  is  not  a straightforward  requirement  but  involves  several  sets  of  vari- 
ables that  must  be  taken  into  account  by  the  human  factors  specialist.  On 
purely  technical  grounds  (which  is  rarely  the  case)  the  issue  is  one  of  the 
extent  of  simulation  and  the  amount  of  fidelity  of  simulation  to  achieve 
certain  training  capabilities  . The  definition  of  the  purpose  of  the  training 
device  figures  prominently  in  tradeoff  decisions  . Purpose  proyides  various 
"shadings  "to  what  is  meant  by  training  value,  and  a number  of  issues  nec- 
essarily come  into  play.  Foremost  are  the  following:  the  transfer  of  train- 

ing situation  (i  . e . , one-to-one  correspondence  with  specific  equipment 
suites  vs  . generalized  equipments);  the  type  of  training  system  and  the  skill 
requirements;  the  levels  of  training  anticipated;  and  the  input  population  of 
trainees  . 


One  feature  should  be  made  clear:  strict  adherence  to  a costs 

schedule  may  violate  a design  principle:  that  of  providing  a design  which 
will  allow  the  widest  rang e of  training  to  be  employed  consistent  with  train- 
ing purpose  . Obviously  , at  this  stage,  the  design  team  is  not  clairvoyant 
in  all  that  should  be  represented  for  training.  The  need  for  flexibility  in 
design  to  maximize  the  positive  transfer  of  training  to.  the  operational 
world,  however  , is  well  understood.  Thus  , one  must  be  wary  of  extreme 
cost  consciousness  for  often , the  purchase  of  additional  simulator  capabi- 
lity may  provide  added  flexibility  in  employing  the  device  for  training, 
which  in  the  long  run  may  bette  r account  for  things  ?uhfo  re  seen  during,  the 
design  stage , hence  , yield  a lower  total  cost  in  terms  of  minimizing  re- 
quired modifications  to  the  device  one e it  is vdn  the  field  and  operating. 
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Research  has  been  unable  to  produce  systematic  techniques  for 
defining  training  value  achievable  in  training  devices  vis-a-vis  dollars 
expended  for  hardware  and  software.  No  body  of  information  (data,  tech- 
niques) is  available  which  provides  the  necessary  guidance  for  achieving 
tradeoffs  between  cost  of  equipment  and  training  value  so  far  as  selecting/ 
omitting  equipment  which  will  govern  the  extent  of  the  training  capability 
(i.e.  , range  or  number  of  tasks  to  be  trained)  . Data,  however,  are  avail- 
able for  determining  ways  of  representing  tasks  so  that  a satisfactory  level 
of  transfer  of  training  can  be  achieved  but  with  reductions  in  costs.  This 
refers  both  to  backing  off  from  complete  (high)  fidelity  of  representation 
and  to  deliberate  departures  from  reality  to  achieve  training  goals.  These 
issues  are  explored  throughout  Section  III  of  this  report. 

To  be  sure,  sophisticated  ways  of  cost/effectiveness  accounting  for 
optimizing  training  system  design  have  been  attempted.  One  attempt  to 
formalize  this  accounting  was  a Naval  Training  Device  Center  program 
which  applied  systems  analysis  techniques  to  the  determination  of  training 
equipment  requirements.  A procedure  was  developed  for  the  ranking  of 
tasks  within  a system  in  terms  of  the  payoff  of  task  training  (as  reflected 
in  improved  system  operations)  per  training  equipment  dollar  expended 
(van  Albert,  Jeantheau,  Gorby  and  Parrish,  1964(a)  , 1964(b)) . This  , called 
the  Training  Analysis  Procedure , was  tailored  to  fit  as  a step  in  NTDC's 
training  requirements  determination  process  (called  the  Training  Situation 
Analysis)  which  culminates  in  a Military  Characteristics  (MC)  Document 
for  the  training  device  under  consideration.  The  procedure  begins  with  a 
system  description  and  an  identification  of  the  tasks  to  be  performed  in  the 
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The  second  step  is  to  collect  performance  data  and  estimate  performance 
for  each  task  before  and  after  training  (via  interviews,  observation  and 
published  data).  Estimates  are  obtained  for  each  task  in  the  system  with 
respect  to  the  probability  that  the  function  will  be  performed  successfully 
and  also  for  the  rate  of  performance.  This  is  followed  (step  3)  by  a speci- 
fication (accomplished  in  conjunction  with  engineers)  of  the  equipment  re- 
quired for  training  each  task  and  the  cost  required  to  provide  that  training. 
From  this,  step  4 is  accomplished,  wherein  cost-effectiveness  ratios  are 
computed  for  each  task  in  the  system  in  order  to  determine  benefit  to 
system  performance  of  training  each  task  in  the  system  and  to  determine 
the  relative  payoff  in  benefit  per  dollar  for  training  each  task.  The  effec- 
tiveness (benefit)  portion  of  this  expression  is  the  figure  of  merit  (FOM), 
which  is  the  percent  estimated  System  improvement  with  training  on  that 
task.. 


For  each  task,  a ratio  is  obtained  defining: 


FOM 


Cost  of  achieving  the 
improvement 

These  computations  are  carried  out  on  an  iterative  basis.  The  task  with 
the  highest  FOM/ Cost  ratio  (i.e.  , greatest  benefit  per  dollar)  is  considered 
’’trained”  on  succeeding  iterations.  The  iterations  proceed  until  all  tasks 
in  the  system  have  been  ’’selected”  in  order  of  their  individual  benefit  to 
system  performance  with  training.  The  outcome  of  this  procedure  is  a 
ranking  of  tasks  in  order  of  their  benefit  to  system  performance  per  dollar 
of  equipment  cost.  Results  are  plotted  as  a step  function,  relating  FOM 
to  Cost  on  a task-by-task  basis.  The  plot  is  a graphic  representation  of 
where  to  put  the  emphasis  in  the  trainer. 


An  interesting  point  in  the  above  technique  is  the  determination  of 
total  costs  of  equipment  per  task.  A system  was  selected  on  which  the 
training  analysis  procedure  was  applied.  This  was  the  Marine  Tactical 
Data  System  (MTDS) , and  a cost  pricing  was  accomplished  (Bertin,  1965). 
The  engineer  who  accomplished  the  pricing  did  it  within  the  usual  costing 
framework  employed  by  NTDC.  The  difference  was  that  the  cost  estimates 
were  made  on  each  task  whereas  the  normal  procedure  costs  are  computed 
on  the  overall  trainer.  The  procedure  for  costing  is  outlined  as  follows: 


O 

ERIC 


Block  diagrams  to  implement  the  training  device 
by  major  components  . 

Schedule  of  events  from  time  of  contract  award  to 
time  item  is  accepted  and  delivered  to  the  particular 
site.  ’ \-S'  \V  ; ;; 


Types  and 


numbers  of  engineers  required  to  design. 


develop  arid  fabricate  the  hardware  components. 


oo 
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• Development  of  work  chart  for  calculating  costs  of: 


engineering  labor 
- manufacturing 
material 

overhead  and  profit 


2.5.  1.  Research  Issues.  It  is  obvious  that  little  in  the  way  of  formal 
approaches  exist  for  cost/ training  value  determinations.  Engineering 
costing  today  accounts  crudely  for  training  value,  i.e.  , only  in  terms  of 
training  system  completeness.  Thus,  differences  in  cost  reflect  in  the 
capability  for  training  in  terms  of  fidelity  (for  example,  omission  of  classi- 
fication cues  in  sonar  operations;  omission  of  IFF  fade  in  MTDS) , or  con- 
trol of  training  (for  example,  omission  of  automatic  performance  evaluatior) 

The  Training  Analysis  Procedure  v outlined  earlier,  has  enough 
limitations  to  obviate  its  general  use.  It  is  limited  to  systems  having  de- 
finable quantitative  outputs.  Its  task  definition  is  not  satisfactory  in  terms 
of  consistency  in  identifying  activities  comprising  a task  and  in  accounting 
for  task  difficulty  levels;  and  it  is  too  crude  in  computation  to  account  for 
the  complexities  of  task  behaviors  that  are  modified  during  the  course  of 
training.  At  best,  it  is  what  the  authors  suggest--an  initial  attempt  to  re- 
move some  of  the  intuition  from  the  results  of  training  requirements 
analysis  . 

What  is  needed  urgently  are  formal  approaches  for  determining  in 
standard  ways,  training  value  received  for  equipment  costs  expended.  Thes 
should  perhaps  be  based  on  operations  research  methods  and  data  and  ex- 
pressed in  mathematical  models.  Allocation  and  utility  theory  should  be 
examined  for  relevance. 


2.6  PHASE  5:  CHARACTERISTICS  OF  THE  OPERATIONAL 

ENVIRONMENT  TO  SIMULATE  FOR  TRAINING.  The  analyses  thus  far 
have  provided  the  basis  for  defining  that  portion  of  the  operational  system 
to  be  simulated  and  organized  as  a training  system.  A number  of  key 
decisions  have  now  been  made;  . " 

• Purpose  of  training  articulated  and  the  basic  fidelity  of  simulation 
requirements  described. 


• Training  objectives  identified  and  organized. 

• Task  structure  defined  and  organized. 

. Equipment  complexes  identified  and  the  fixed  components 


set  forth. 
Overall  training 


and  operational  modes  roughed  out 
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The  human  factors  effort  now  concentrates  on  selecting  the  design 
alternatives  which  will  define  the  manner  in  which  the  synthetic  environ- 
ment for  training  will  be  provided  and  how  the  shaping  of  learning  (structure 
and  control  of  training)  will  be  accomplished.  The  human  factors  contri- 
bution to  design  embraces  two  distinct  phases:  1)  the  fidelity  of  simulation 

issues  in  representing  the  operational  environment  in  the  trainee  compart- 
ment) s)  , and  2)  the  management  of  training  which  concerns  the  monitoring  , 
evaluation  and  control  of  training  at  the  instructor  station. 

This  much  should  be  made  clear  at  the  outset:  translating  behavioral 

information  into  device  design  is  not  a straightforward  operation.  It  is 
difficult  for  several  reasons.  1)  The  state-of-the-art  in  training  technology 
has  not  advanced  to  the  position  where  transfer  of  training  relationships 
are  sufficiently  understood,  nor  does  a body  of  "training  value"  data  exist 
that  correlates  well  with  design  alternatives.  2)  Synthetic  representation 
(i.e.  , engineering  capability)  is  not  easily  or  completely  achievable  in 
every  case.  Compromises  and  less  than  desirable  representation  are  used 
as  the  "best  available"  in  certain  areas.  3)  Since  100%  fidelity  is  not  al- 
ways possible  (nor  in  some  instances,  desired)  , the  extent  of  backing  off 
from  this  and  not  sacrificing  training  value  in  both  operational  and  motiva- 
tional terms  is  crucial  to  effective  human  factors  design.  4)  There  are 
alternative  ways  to  "encourage"  trainees  to  accept  simulation  as  highly 
faithful  to  the  operational  world.  Different  tradeoff  values  are  involved 
with  each  and  the  available  design  options  must  be  identified  and  assessed 
carefully.  5)  The  values  of  deliberate  departures  from  realism  ( deviations 
in  configuration/ operation  from  the  operational  system  being  simulated)  to 
enhance  training  effectiveness  must  be  clearly  understood  and  effectively 
implemented. 

To  most  effectively  achieve  the  simulation  requirements  and  fidelity 
levels,  it  is  a desirable  habit  to  view  the  design  implications  (hardware 
orientation)  in  terms  of  the  training  requirements  . Decisions  on  what  must 
be  simulated  and  the  selection  of;  design  alternatives  depend  substantially  on 
the  purpose  of  training,  the  training  objectives , the  task  structure  and  the 
simulation  options  available  within  cost  and  state-of-the-art  constraints. 

An  interrelated  revolving  set  of  i~elationships  is  at  work  in  which  each  incre- 
ment in  the  functional  design  of  the  training  system  is  related  back  to  the 
training  requirements  information.  The  relationships  look  something  like 


the  following: 
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Answers  to  these  kinds  of  questions  are  meaningful  only  in  light  of  what 
training  is  supposed  to  accomplish,  what  people  are  involved  and  what  they 
will  do  in  training,  what  the  job-task  structure  is,  and  how  training  will 
vary  as  a function  of  the  design  alternatives. 

The  design  criterion  is  clear:  provide  the  means  for  exercising  and 

developing  the  human  performances  required  in  the  operational  environment. 
How  to  achieve  this  is  not  always  clear.  A system  of  general  principles  for 
the  design  of  synthetic  training  systems  does  not  exist  in  sufficient  detail  to 
provide  the  necessary  support  to  human  factors  design.  Nor  is  an  invariant 
series  of  design  principles  always  useful.  There  is  a considerable  number 
of  different  kinds  of  training  devices  currently  in  use.  Most  of  these,  es- 
pecially the  complex  synthetic  systems  are  built  in  limited  numbers.  In 
fact,  it  is  meaningful  to  say  that  each  important  and  complex  training  system 
is  a research  and  development  effort  in  its  own  right.  However,  most  train- 
ing systems  can  be  grouped  into  a small  number  of  device  classes,  each 
class  possessing  characteristics  common  among  its  members  . Within  a 
class,  there  is  a series  of  design  principles  that  hold  for  any  member  of 
that  class.  The  approach  taken  in  this  method  is  to  provide  guidance  by 
organizing  and  developing  design  issues  common  to  classes  of  training 


devices . 


Three  major  classes  of  devices  are  identified,  vis-a-vis  training 


purpose 


Device  Class 


Primary  System  Characteristics 


Tactical  Systems 


(Complex  multi-man  airborne, 
surface  and  sub- surface  trainers 
emphasizing  tactical  team  training) 


Own  vehicle  and  support  units  char- 
acteristics 

Target  characteristics 

Multiple  sensor  classes  and  displays 
Vehicle  position  geometry 
Environmental  characteristic s 
Weapons  characteristics 


Operational  Aircraft  Systems 


(Fixed  wing  and  rotary  wing 
Operational  Flight  Trainers  and 
Air  Weapon  System  Trainers) 


Real-world  visual  simulation 


Instrument  displays 
Vehicle  motion 
Sensor  displays 


V ehic  le  dynamic  s 


Weapons  characteristics 
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Device  Class Prime  ry  System  Characteristics 

Individual  Operator  Trainers 

(Specific  part-task  training  devices,  Procedural  events  fidelity 

generalized/univer  sal  training  Instrument/ sensor  displays 

devices)  Weapons  control  characteristics 

Target  characteristics 


Many  issues  must  be  considered  in  arriving  at  a description  of  the 
training  system  under  development.  Some  obviously  are  peculiar  to  a sys- 
tem and  are  generally  a one-shot  solution  to  the  problem  at  hand;  a number 
of  others,  however,  are  more  basic  and  are  involved  in  the  development  of 
most  training  systems  or  in  the  development  of  a class  of  training  system. 
These  latter  issues,  the  numbers  of  recurring  and  fundamental  design 
problems  in  the  development  of  most  synthetic  trainers,  are  of  prime  in- 
terest to  this  guide  and  receive  priority  emphasis. 

To  place  the  key  design  issues  in  perspective,  the  human  factors 
inputs  in  the  specification  of  functional  design  are  subsumed  under  two 
major  and  interrelated  categories: 

• Representing  the  operational  environment  in  the  trainee 
compartment)  s)  . 

• Providing  for  the  management  of  training- -structuring, 
monitoring,  evaluating  and  controlling  training  at  the 
instructor  station. 

Achieving  the  design  options  available  in  specifying  the  functional 
.'  characteristics  for  the  training  system  requires  that  a number  of  questions 
be  formulated,  phrased  so  as  to  consider  the  relationship  with  training  pur- 
pose and  task  structure.  These  design  issues  will  form  the  basis  for  the 
presentation  here . What  the  human  factor s specialist  must  consider  is 
documented  here  and  ranges  from  relatively  simple  concerns  about  items 
that  must  not  be  overlooked  in  the  design  process,  to  complex  design  issues 
involving  a design  choice  based  on  reasonable  alternatives.  The  issues 
underlying  each  design  question  and  the  design  alternatives  or  options  for 
solution. will  be  identified.  In  Section  III,  specific  and  quantitative  informa- 
tion from  the  research  literature  is  presented  in  support  of  these  issues  . 

In  addition,  research  needs  are  defined  for  those  instances  where  human 
factors  design  data  are  incomplete  or  not  available. 

2.7  PHASE  5 A:-  REPRESENTING  THE  OPERATIONAL  ENVIRONMENT 
IN  THE  TRAINEE  STATION(S).  So  far  as  organizing  the  environment  in 
, which  the  learning  will  take  place  is  c one  erne  d , the  central  issue  is  , fidelity 
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of  simulation.  For  human  factors  , fidelity  has  meaning  in  terms  of  the 
training  process  and  the  realism  necessary  to  promote  transfer  of  training. 
Defining  the  design  characteristics  for  maximizing  transfer  of  training  from 
the  synthetic  environment  to  the  operational  requirement  revolves  essen- 
tially about  two  interrelated  questions:  what/how  much  should  be  simulated? 

and  how  well  should  this  be  represented? 

V 

Trainee  compartment  design  considers  most  prominently  the  issues 
involved  in  achieving  simulation  fidelity,  that  is,  specifying  the  hardware 
(displays /controls)  involved  in  the  device  and  the  fidelity  levels  required 
at  the  man-machine  interfaces,  ranging  from  high  fidelity  in  engineering 
simulation  to  deliberate  departures  from  reality  involving  special  features 
of  design  not  found  in  the  operational  system,  in  order  to  enhance  learning. 

To  place  trainee  compartment  design  in  perspective,  the  range  of 
human  factors  issues  and  design  alternatives  are  subsumed  under  two  ■ 
major  categories:  ... \ > 


• Configuration  of  the  synthetic  system. 

• Simulation  elements  and  fidelity  of  simulation. 

The  issues  discussed  are  not  necessarily  of  similar  importance  in  terms  of 
training  value  or  of  the  human  factors  effort  required,  nor  is  there  a re- 
curring similarity  in  the  way  each  issue  is  handled  and  resolved.  As  indi- 
cated earlier,  the  approach  calls  attention  to  the  range  of  human  factors 
issues  involved  in  trainee  compartment  design  and  this  provides  the  reader 
guidance  in  selecting  those  topics  pertinent  to  his  specific  needs. 

2.7.1  Synthetic  System  Configuration.  The  job  at  this  point  is  to  flesh- 
out  the  system  components  involved  in  the  trainee  compartment.  The  need 
for  specific  simulation  capabilities  is  now  explored  based  on  the  groundwork 
laid  in  the  previous  phases  i To  be  resolved,  are  those  portions  of  the  sub- 
systems that  must  be  represented  that:  significantly  affect  training  capability. 
In  many  devices,  particularly  multi-man  tactical  trainers,  direct  transfer 
of  training  is  desired  from  device  to  the  operational  vehicle , hence;  replicas 
of  consoles  and  equipments  are  desired  in  the  compartments.  Similarly, 
OFT/WST's  rely  on  high  fidelity  cockpit  mockups  . Certain  generalized 
trainers  provide  training  across  a range  of  specific  systems  (e.  g.  , device 
15E18,  or  the  Air  Force  Simulator  for  Electronic  Warfare  (SEWT),  but 
nevertheless  require  high  fidelity  in  the  specific  E¥  equipments  so  that,  the 
signal  signatures  achieved; are  equivalent  to  those  in  the  operational  environ- 
ment so  far  as  the  trainee  is  concerned.  Thus,  decisions  are  made  on  the 
"core"  equipments  needed  to  operate  analogous  to  the  real  world . What  1 . j. 

instruments /equipments  are  operable  replicas,  what  are  operable  non- 
replicas and  what  are  non-operable  replicas  depend  on  training  purpose. 


The  fleshing-out  (i.e . , defining  what  -is  .to  be  simulated)  of  the 
configuration  considers  the  following: 


ERIC 
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• Appearance  and  operation  of  key  equipment  complexes  (controls 
and  displays)  in  trainee  compartments  for  pertinent  subsystems. 


Compartment/ cockpit  configuration 
Real-world  visual 
Platform  motion 
Sensors 
W eapons 

Specific  capabilities,  vis-a-vis  training  purpose 
(e.g. , inflight  refueling) 


• Exact  representation  of  operational  equipment  (fixed  components 
central  to  system  operation)  for  those  portions  of  the  system  to 
be  simulated  ( e . g . , communications /navigation  facilities  such 
as  radio,  TACAN,  Doppler  Radar  Navigation;  relevant  vehicle 
instruments;  periscope  configuration,  etc;). 

m Functional  representation  of  fixed  components  to  provide  in- 
formation inputs  on  discrete  events,  where  these  equipments 
are  related  to  the  real  system  only  in  an  output  or  function 
performed,  but  not  in  appearance  or  operation. 


• Extra  mission  equipments  (e.  g.  , for  demonstration,  for  trainee 
selection  of  an  exercise,  etc.). 

In  tactical  team  trainers  where  the  training  emphasis  is  on  coordi- 
nated team  procedures  and  the  tactical  employment  of  friendly  vehicles 
(own- ship  and  support  units)  , the  design  practice  is  to  provide  replicas  of 
equipments  and  consoles  in  the  compartments  (i.e.  , duplication  in  appear- 
ance and  operation  of  key  equipment  complexes  and  exact  representation  of 
communications  equipments  and  ancillary  displays  and  controls  such  as 
ship’s  instruments)  . This  is  logically  sound  in  that  direct  transfer  of 
training  is  desired  since  these  devices  are  used  in  direct  support  of  line 
vessels.  Team  practice  in  the  device  should  be  highly  realistic  of  actual 
shipboard  operations  with  system  suites  equivalent  to  those  found  aboard 
ship,  and  in  similar  configurations. 

Design  practices  for  operational  flight  trainers  rely  on  high  fidelity 
cockpit  mockups.  Decisions  on  operable  instruments  and  equipments  are 
based  on  defined  normal  operating  sequences  and  emergencies  to  be  trained. 
The  selection  of  active  instruments  should  be  agreed  upon  with  the  using 
agency  for  the  device  . The  selection  of  operational  components  , however  , 
must  take  into  account  future  modification  possibilities  and  training 
flexibility  ■ : - 


Thus;,  for  devices  representing  an  operational  system,  a 
of  all  equipments  pertinent  to  the  compartment  is  made;  for  generalized/ 


■ ' 0 

ERIC 


32 


.■A*-.-  'KP 


81  - 


NAVTRADEVCEN  69-C-0298-1 


universal  trainers,  a listing  of  all  functions  to  be  performed  is  made  and 
the  simulation  elements  then  determined  to  achieve  these  functions. 

The  employment  of  actual  system  equipments  (e.  g.  , Government 
Furnished  Equipment)  vs.  simulated  equipments  is  resolved  here,  but  for 
the  most  part,  this  is  an  engineering  issue  which  considers  the  availability 
of  operational  equipment,  maintenance  (ruggedness/ cost)  and  budget/cost 
factors.  The  human  factors  input  is  in  the  analysis  of  possible  discrepan- 
cies in  fidelity  when  using  simulated  equipments,  and  in  deliberate  modi- 
fications in  the  information  displayed  on  operational  equipment  to  enhance 
training  effectiveness. 

2.7.2  Simulation  Elements.  Once  the  configuration  of  the  device  is  set, 
the  issues  of  specifying  those  parameters  which  define  ?,how  well"  simula- 
tion is  achieved,  are  resolved*  Specifying  the  fidelity  of  simulation  is  the 
crucial  requirement  in  representing  the  synthetic  environment  in  which 
training  is  conducted.  The  human  factors  input  concerns  the  definition  of 
the  simulation  elements.  These  represent,  in  hardware  terms,  the  para- 
meters and  their  values  (range,  envelope,  number)  needed  to  accomplish 
the  desired  training,  hence  govern  the  range  and  complexity  of  task  installa- 
tion in  the  device.  The  simulation  elements  which  are  controllable  (mani- 
pulated or  modified  manually  or  automatically  via  the  instructor  station) 
determine  the  perceptual  equivalence  of  the  training  environment  and  the 
operational  situation.  The  ability  to  provide  the  desired  training  in  the 
identified  tasks  is  directly  dependent  on  the  availability  and  adequacy  of 
these  simulation  elements  in  a training  device  (i.e.  , which  ones  are  selec- 
ted and  how  usefully  they  are  represented).  Shortcomings  in  the  simulation 
of  these  elements  define  the  shortcomings  in  the  training  capability  of  a 
device  (i.e.  , simulation  elements  are  associated  with  the  representative- 
ness/complexity of  the  tasks  to  be  trained  in  the  device).  The  proverb, 

Mfor  the  want  of  a nail,  a shoe  was  lost.  . . n is  the  meaning  implied  here. 
Example:  failure  to  provide  airborne  interceptor  threats  in  an  airborne 

EW  trainer  delimits  the  range  of  training  in  establishing  threat  priority 
and  conducting  defensive  maneuvers;  example:  failure  to  provide  ASW 

destroyer  sonar  detection  ranges  in  excess  of  10, 000  yards  compromises 
training  in  tactical  decision  making  employing  long  range  weapons  (such  as 
ASROC,  DASH);  example:  failure  to  provide  sonar  classification  cues  de- 

limits the  capability  of  an  ASW  team  trainer  (e.  g.  , ASROC / ASW  Early 
Attack  Weapon  System  Trainer,  Device  14A2)  to  conduct  tactical  employ- 
ment of  a fighting  ship  in  its  full  meaning.  Thus,  adequate  specification  of 
these  elements  is  required  so  as  to  insure  that  training  purpose  and  train- 
ing objectives  are  achieved  ( thereby  enhancing  the  transfer  of  training 
potential).  Since  these  elements  are  manipulatible,  they  also  affect  the 
excellence  of  the  utilization  context  for  the  sequencing  of  training  (again, 
in  terms  of  purpose  and  training  objectives)  . In  short,  the  simulation 
elements  define  what  is  needed  in  the  task  environment  to  achieve  the  ex- 

effective  utilization  of  the  device. 
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These  controllable  elements  refer  to  the  training  mission  environ- 
ment at  the  trainee  compartment  and  narrow  down  to  three  major  groupings 
of  characteristics:  own-vehicle  and  support  units;  targets;  and  the  media . 

Within  each  of  these  major  groupings,  the  following  subsystems  are  involved: 
sonar,  radar,  electronic  warfare,  visual  real  world,  vehicle  dynamics, 
and  weapons  . The  matrix  outlining  the  interrelationships  has  already  been 
shown  in  Figure  11. 

2.7.2. 1 Levels  of  Fidelity 


Five  levels  of  simulation  fidelity  are  pertinent  to  the  human 
factors  specification  of  the  characteristics  required  to  insure  effective  task 
installation  for  the  training  system  under  consideration.  How  well  each 
simulation  element  is  to  be  represented  for  training  should  be  evaluated  in 
terms  of  these  five  levels  or  dimensions.  Table  5 describes  the  fidelity  of 
simulation  levels  used  in  evaluating  each  of  the  simulation  elements  identi- 
fied within  the  categories  in  Figure  11.  For  level  1,  the  decision  is  to 
either  select  or  exclude  the  element  from  simulation.  Based  on  the  decision 
to  represent  an  element  in  design,  the  following  dimensions  are  then  exam- 
ined to  determine  the  fidelity  required  to  be  useful  for  training.  For  level 
2,  the  decision  is  to  select  the  required  value(s)  for  the  element  (for  exam- 
ple, six  levels  of  sea  state  for  sonar  operations);  for  level  3,  the  decision 
is  in  terms  of  engineering  precision  (for  example,  EW  signal  signatures  for 
an  airborne  electronic  warfare  trainer);  for  level  4,  the  decisionr'is  in  terms 
of  a necessary  reduction  in  tolerances,  or  a backing  off  frqm 'engineering 
fidelity  (for  example,  radar  land  mass  depiction  at  low  altitudes,  or  elimina- 
tion of  a term(s)  in  the  mathematical  model  for  aerodynamic  simulation); 
and  for  level  5,  the  decision  deals  with  special  features  of  the  training  sys- 
tem and  concerns  the  desirability  of  deliberately  deviating  in  the  design  of 
operational  equipments  , other  than  on  the  engineering  fidelity  continuum, 
to  enhance  transfer  effects  (for  example,  employing  a stimulus  enhancement 
technique  on  a primary  visual  display,  or  displaying  alpha-numeric  perform- 
ance information  (augmented  feedback)  directly  on  a student’s  CRT  display)  . 


2.7.3  Classification  of  Simulation  Elements.  The  whole  array  of  the 
simulation  elements  that  must  be  evaluated  for  representation  in  the  design 
of  the  trainee  compartment  to  achieve  control,  display  and  procedural  re- 
quirements, is  presented  next.  These  elements  of  simulation  are  organized 
in  terms  of  own- ship  and  support  units  , target,  and  media  characteristics  . 
For  each  class  of  task  element  within  each  of  these  three  groupings  , deci- 
sions must  be  made  about  various  dimensions  of  fidelity  required.  To  aid 
in  this  determination,  the  five  levels  for  evaluating  fidelity  of  simulation, 
identified  in  Table  5 are  used.  Thus , in  the  following  outline  ( Table  6),  the 
classes  of  simulation  elements  are  grouped  within  major  subsystems  appli- 
cable to  the  device  under  consideration.  This  is  accomplished  (as  applicable) 
independently  for  each  of  the  simulation  areas  (own-ship,  target,  media) 
since  the  classes  of  simulation  elements  are  different  for  each  area.  Thus, 
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TABLE  5.  FIDELITY  OF  SIMULATION  LEVELS  FOR  EVALUATING 
SIMULATION  ELEMENTS 


Level 

Description 

i 

Indus  ion/ exclusion  of  the  element. 

2 

Representative  envelope/ steps/ value  for  the  element. 

3 

Degree  of  fidelity  required  (where  engineering  state-of-the- 
art  is  adequate)  . . .. 

4 

Fidelity  achievable  where  reduction  in  tolerances  is  re- 
quired  (where  engineering  state-of-the-art  is  less  than 
adequate) . 

5 

Deliberate  departures  from  realism  to  enhance  training 
effectiveness  (deviations  in  configuration/ operation  associ- 
ated with  the  operational  system  being  simulated) . 

for  example , the  classes  of  simulation  elements  for  the  sonar  subsystem  are 
identified  for  own-ship  and  support  units,  again  for  target  characteristics, 
and  again  for  the  simulated  medium.  The  appropriate  fidelity  dimension(s) 
to  consider  is  indicated  (identified  by  corresponding  number  in  Table  5)  for 
each  class  of  simulation  element  in  each  subsystem  for  each  of  the  three 
simulation  areas  . 


2.7.4  Output.  Identification  is  made  of  the  crucial  factors  in  representing 
the  operational  environment  in  the  trainee  compartment.  The  functional 
characteristics  of  trainee  compartment  design  consider  most  prominently 
the  issues  involved  in  achieving  simulation  fidelity  in  creating  an  environ- 


ment for  learning 


0 

ERIC 


The  completion  of  this  phase  yields  the  following: 


Configuration  of  the  synthetic  system  (in  terms  of  the  extent 
of  representation,  the  hardware  components  in  the  trainee 
compartment,  and  specific  equipment  requirements  . 


Definition  of  the  task  structure  arid  definition  of  simulation 
■!  elements  (parameters  to  be  simulated,  which  govern  the  range 
and  complexity  of  task  representation  in  the  device  and 
which  are  required  to  achieve  training  purpose  and  the 
• training  objectives)  . ; ! ^ • 


m 
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TABLE  6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS 


I.  OWN -VEHICLE  AND  SUPPORT  UNITS 

A.  SONAR  (Electronic  Visual  Symbol  and  Auditory  Display) 

1.  OWN-VEHICLE  RETURNS 

(fidelity  dimens  ions  (3)  ) 

. • 

own-ship  noise  (flow 

appropriate  video/ audio 

noise  as  a function  of 

characteristics 

speed,  turbine  noise, 
steam  noise,  noise 

appropriate  video/audio 

associated  with 

degradation  as  a function  of 

weapon  actions  , in- 

speed  increase  , correlated 

ternal  noises  of  crew, 
pumps) 

with  optimum  sonar  speed 

• 

wake 

• 

knuckles 

• • 

torpedo  hydrophone 
effects 

• 

countermeasures 

• " 

cavitation 

own- ship  cavitation  sounds 

(depth/ speed) 

detectable  when  cursor  is 
placed  in  baffles  area 

2.  SUPPORT  UNIT  RETURNS 

(fidelity  dimensions  ) 

• 

support  ship(s) 

Video  - appropriate  blip  size. 

(aspect) 

shape,  brightness  for  the 
vehicle,  correlated  with 

• 

effects  of  echo- 

aspect 

ranging  (mutual 

';V.  '••  • ' L.;.':-.  'L:  .... 

interference  effects 

Audio  - appropriate  echo 

from  surface  ships , 

quality,  doppler  effects  , , 

dipping  helos  , sono- 

correlated  with  aspect 

buoys  , submarines) 

. •• ; ; ■;  •; ' i 

pings  from  other 
helicopters  / 
submarines  - : 

; •• ' V;--.: ' . ...  !•  '■  - iyu  : L ’ i ! ' 

O , 
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6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 


• noise  (e.g. , garbled 
noise  from,  under - 
wate r tele pho ne) 

MALFUNCTION  CAPABILITIES 
IN  EQUIPMENT 

. • audio  failure 

• video  failure 

• automatic  target  following 
(ATF) 

• recorder  failure  (bearing 
information  trace) 

• own  ship  course  informa- 
tion 

. tape  recorder  (e.g.  , in 
submarine  for  turn  count) 

• BT  layer  information 
(submarine) 


loss  of  bug  (manually 
trained- submarine) 

bearing  position  informa- 
tion 


(fidelity  dimension  (Jv  ) 


PPI  DISPLAY  C HA  RAC  TER- 
ISTICS  - PERCEPTUAL 
REQUIREMENT  S 

• echo  quality 

. pip  quality  (aspect) 

• axis  angle  % ;• 

. - tr ac  e le ngth  '?  ■ 

. differential  range  rate 


(fidelity  dimension  ) 


Of 
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TABLE  6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 


5.  VISUAL  DISPLAY 
REQUIREMEN  TS 


(fidelity  dimension  ®> 


. acuity 
. luminance 


refresh  rate  (flicker) 


contrast  sensitivity 


resolution 


6.  TRANSFER  OF  SONAR  (fidelity  dimensions  CD®) 

DATA  TO  OTHER 
SUBSYSTEMS 

(e  . g.  , to  fire  control  subsystem) 


. range,  speed, ^bearing, 
target  course  (manual 
insert) 


. generated  target  tracking 


7.  DELIBERATE  DEPARTURES  (fidelity  dimensions  ©) 
FROM  REALISM 


display  of  performance  information  (knowledge  of  results) 
to  trainee  (on  primary  display  or  via  separate  equipment) 


• coding  of  signals  (e.g.  , alpha-numeric) 
. stimulus  enhancement 


B.  Rj-.DAR  (Electronic  Visual  Symbol  Display) 


1 . 


OWN- VEHICLE  RETURNS  (fidelity  dimensions  ) 

. reflection  echoes  (surface  ship)  £ 


wake  (Surface  ship) 


super  s t r uctur e 
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TABLE 

6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 

2. 

; . . v.  . . . . . . 

. interference  from  other  radars  in  own  vehicle' or  in 
proximity  to  own  vehicle  ; 

• own  vehicle  depth/ antenna  height  above  water 

SUPPORT  UNIT  RETURNS  (fidelity  dimensions  ©(3)  ) 

. surface  ships,  aircraft  1 blips  proportional  in  size,  shape 

and  brightness  to  vehicle  type, 
aspect,  range  and  range  scale 
selected 

3. 

• wakes  (surface  ships)  | appearance  and  definition 

| correlated  with  ship  speed 

PPI  DISPLAYS  - perceptual  (fidelity  dimensions  ) 

requirements  ... 

• range  strobe 

• sweep  line 

. offset  features  . . s - . . ’ 

, target  designation  features  (e  . g,  , Hooking)' 

4. 

• enhancing  the  signal  (in  operational  equipment) 

SELF- TEST  INDICATIONS  (fidelity  dimensions  (h)(2)  ) 

5. 

VISUAL  DISPLAY  (fidelity  dimensions  (3)  ) 

REQUIREMENTS 

■■Vi'  acuit\ 

, luminance 

• refresh  rate  (flicker) 

• contrast  sensitivity  ■ . 

• resolution  ']■[  r-'* 
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TABLE  6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 


6.  MALFUNCTION  CAPABILITIES  (fidelity  dimension  (7)  ) 
IN  EQUIPMENT 

. partial  failures 

. degrading  of  resolution  (e  . g . , range,  video) 


• transmitter  loss 

7.  DELIBERATE  DEPARTURES 
FROM  REALISM 


(fidelity  dimension  (5)  ) 


display  of  performance  information  (knowledge  of  results) 
to  trainee  (on  primary  display  or  via  separate  equipment) 


. coding  of  signals  (e.g.  , alpha -numeric; 

- what  symbology 

- how  many  codes  needed 
. stimulus  enhancement 


ELECTRONIC  WARFARE  (Electronic  Visual  Symbol  Display) 
1.  OWN- VEHICLE  RETURNS  (fidelity  dimension  (5)  ) 


noise,  interference 
board,  placement  of 
harmonics) 


effects  (e.g.  , ECM  emitters  on 
antennas  aboard  surface  vehicle, 


. electromagnetic  interference  (EMI)  from  on-board 
equipment 


2.  SIGNAL  CHARACTERISTICS  - (fidelity  dimension  \ 3)  ) 
DISPLAY  PERCEPTUAL 
REQUIREMENTS 

• pulse  frequency  (modulation)  : 

. pulse  repetition  frequency  (PRF)  \ ’ 

. peak  power  (range  between  transmitter  and  receiver) 


: ■'*3  ■ 
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TABLE  6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 
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TABLE  6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 


(fidelity  dimension  ) 


6.  VISUAL  DISPLAY 

REQUIREMENTS 

• acuity 

• luminance 
refresh  rate  (flicker) 

• contrast  sensitivity 

• resolution 

7.  DELIBERATE  DEPARTURES  (fidelity  dimension  (?)  ) 

FROM  REALISM 

. display  of  performance  information  (knowledge  of  results) 
to  trainee  (on  primary  displays  or  via  separate  equipment) 

. ■ masking  of  displays  in  universal/generalized  trainers 
not  utilized  in  specific  mission  context 


D.  VISUAL  REAL  WORLD 


(fidelity  dimensions  QXS®®) 


1.  AIR  TO  GROUND  REQUIREMENTS 

• artifacts  to  represent  (buildings  , vehicles,  bridges,  etc.) 

• areas /elevations 

. size  change s/ per spective  (function  of  range /altitude) 

. complexity  of  visual  detail 

. figure-ground  characteristics  (object(s)  on  land/ water) 

. scale  ■ I 
. display  of  weapon  effects 

2.  AIR  TO  AIR  REQUIREMENTS 

. target  characteristics  4 • : • " ; 


; ii- 
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TABLE  6 . CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 
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TABLE  6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 


• visual  angle 

, target  range  ( stadimeter/telemeter) 

m seascape/ landmass  (relative  movement  between  own 
ship  and  seascape) 

• perspective  of  target  images  displayed 

. background/ special  effects  (effect  of  earth  curvature, 
i.  e.  , variable  relationship  between  fixed  horizon  and 
moving  target 

m variable  scope  exposure  as  function  of  keel  depth 
(target  visibility  and  "height  of  eye"  of  observer) 

6.  COMPATIBILITY  OF  VISUAL  REAL-WORLD  SCENE 

• with  movement  of  own-vehicle 

• with  other  displays /instruments 

7.  VISUAL  SCENE  PROGRAMING 
(Programed,  semi-programed,  non-programed) 

8.  TARGET  MOVEMENT  AS  FUNCTION  OF 
RANGE,  BEARING,  ALTITUDE,  ASPECT 
CHANGES 

(fidelity  dimensions  ) 

• type  of  vehicle/ characteristics 

- speed  range  ' ‘ 

- ordered  course 

- acceleration/deceleration  characteristics 

- turn  rate/ radius 


E.  VEHICLE  DYNAMICS 

1.  OWN-VEHICLE 

CHARACTERISTICS 


0 
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TABLE  6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 

- dive /climb  rate 
ordered  depth/altitude 

- vehicle  response  lags 

2.  SUPPORT  UNITS  (fidelity  dimensions  CD©©) 

CHARACTERISTICS 

• type  and  number 

assist  ships  (e.g.  , SAU  destroyer  units;  aircraft 
for  VECTACS) 

- aircraft  (e.g.  , formations,  remote  control  (DASH), 
vehicles) 

- aircraft  carrier(s) 

• vehicle  characteristics 

speed  range 
ordered  course 

- acceleration/ deceleration 
turn  rate/ radius 

- dive/ climb 

- vehicle  response  lags 

3.  RELATIVE  MOTION  OF  (fidelity  dimensions  @@  ) 

VEHICLES  DISPLAYED 

4.  PLATFORM  MOTION  (fidelity  dimensions  ®(D©) 

REQUIREMENTS  . 

. motion  parameters  required  (degrees  of  freedom) 

» values  for  defined  motion  parameters 

• acceleration  tolerances  (onset,  washout  rates) 

lOvvyv 
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TABLE  6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 


„ random  forces  (turbulence,  vibration,  jerk) 

• control/ display  compatibility  (visual-motion 
integration) 

5.  AERODYNAMIC  SIMULATION  (fidelity  dimensions  (D©©> 

. mathematical  modeling  requirements 

- dynamic  vs  . kinematic  (computer  controlled)  models 
(i.e.  , trainee  inputs  in  control  forces  such  as  con- 
trolling an  aircraft,  vs.  trainee  ordering  of  the 
commands  while  not  in  dynamic  response  loop, 
such  as  an  EW  trainee  conducting  an  evasive  maneuver 
in  response  to  an  airborne  interceptor,  i.e,  , he  only 
orders  the  maneuver- -the  simulated  own-aircraft 
completes  the  maneuver  and  automatically  returns 
to  the  previous  track)  . 

definition  and  selection  of  terms  in  the  equation  of 
motion  to  optimize  the  equation,  i.e.  , what  terms 
to  include  (linear  rates  and  velocities,  linear  and 
angular  accelerations) 

. dynamic  control  force  requirements 

aircraft  characteristics  (stick  forces) 

, controls -displays  compatibility 

coupling  of  aerodynamic,  visual  and  motion  functions 

6.  VEHICLE  SOUNDS  (AUDITORY)  (fidelity  dimensions  (J)© ■) 

• ambient  noise  (e.g.  , cockpit  noises,  changes  in  submarine 
configuration,  steam  and  flow  noise  s in  submarine) 

. engine  sounds  (e.g.  , turbine  power,  rotor  engagement 
compressor  stall,  failure  characteristics) 

.m  sounds  related  to  pertinent  events  in  mission  context 

(e.  g.  , tire  screech  on  landing  impact,  turbine  changes) 

7.  OLFACTORY  CUES  (fidelity  dimension s(T)(2^)(3^)) 
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TABLE  6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 


» target  appearance  /disappearance/attenuation 
as  a function  of  range  and  sea  conditions 

• target  movement  as  function  of  range,  bearing. 


aspect  changes 

2.  TARGET  VEHICLE 
RETURNS 

(fidelity  dimensions  ) 

• aspect  (bow,  beam, 
stern) 

Video  - appropriate  blip  size, 
shape  and  brightness  persistence 
for  brightness  modes 

Audio  - doppler  effects  (up,  down, 
no)  via  increase/no  increase/ 
decrease  in  pitch  over  transmis- 
sion frequency 

• cavitation  effects 

Video  - appropriate  noise  spoke 
for  speed,  depth,  range  and  BT 

• hydrophone  effects 
from  torpedo 

envelopes 

Audio  — appropriate  audio  char- 
acteristics  for  speed,  depth, 
range  and  BT  envelopes 

• knuckles 

Video  - appropriate  blip  size, 
shape,  and  brightness  persistence 

• wake  s 

for  brightness  modes 

Audio  - appropriate  audio 
characteristics 

• tonals 

specific  frequency/frequency 
band  which  stands  out  from 
general  increase  in  noise  levels 
on  a bearing 

vehicle  sounds  (screw 
beat  (turns  per  knot 
ratios)  , machinery 
noises,  housekeeping 
noises) 
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TABLE  6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 


3.  DELIBERATE  DEPARTURES  (fidelity  dimension  (?)  ) 
FROM  REALISM 


B . RADAR 


I 


1.  TARGET  GHA EAC  TERISTIC S (fidelity  dimensions  ( XX  2 ) ) 


■ U ' 


. total  number  of  targets  to  represent  in 
mission  environment 

• classes  of  targets  .t 


. number  of  targets  displayed  simultaneously/ 
instantaneously  (number  that  the  trainee  is 
capable  of  processing  in  his  span  of  control) 

. target  appearance/ disappearance/ attenuation 

as  a function  of  range /altitude  and  environmental 
conditions  (e.g. , radar  fades,  height  of  antenna) 

9 capability  for  add-on  of  new  targets  (for  updating) 

TARGET  RETURNS  (fidelity  dimensions  .Cl 


V) 


submarine 

- surfaced 

periscope  depth/ 
snorkel 

- feathers 


aircraft 

speed  and 
mane uve  r s 


blip  proportional  in  size, 
shape  and  brightness  to  vehicle 
type,  aspect,  range  and  range 
scale  selected 

correlated  with  submarine 
speed  at  periscope  depth 


blip  proportional  in  size , 
shape  and  brightness  to  vehicle 
type,  aspect,  range  and  range 
scale  selected 


O 
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TABLE  6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 


vectoring  of 
aircraft  in  IFR 
departure 
(GCA -CCA) 

- relative  motion 

lock- on 
indications 

target  identifi- 
cation ( e . g . , SIF) 

surface  vehicle 

speed  and 
maneuvers 


blip  proportional  in  size, 
shape  and  brightness  to  vehicle 
type,  aspect,  range  and  range 
scale  selected 


3.  LAND  MASS  SIMULATION  (fidelity  dimensions  (T)  @ @ ) 
• characteristics 

smallest  target  size  defined 

radar  modes  defined  •>. 

- aspect  angle/ shadowing 

beam  width  ( vertical/horizontal) 

- scintillation  (change  in  target  brightness) 

, accuracy  of  moving  across  land  mass  display 


evation)  (e.  g.,  ring 
of  city,  form  of 


far  shore  brightness  (el 
of  shoreline  / perimeter 
aspect  angle)  I 

range  of  degradation 


definition  of  gray  steps'  ( altitude  depiction-ability 
to  resolve  object  height) 


O \ 
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TABLE  6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 


4.  TARGET  DISTORTIONS  (fidelity  dimensions  ) 

• garbling  (signal  overlap) 

5.  DELIBERATE  DEPARTURES  (fidelity  dimensions  (T)  ) 

FROM  REALISM  .'..1^.; 

• slaved/ non- dynamic  targets 

C.  ELECTRONIC  WARFARE 

1 . TARGET 

CHARACTERISTICS 


O - 

ERIC 


(fidelity  dimensions  ) 


• total  number  of  emitters  to  represent  in 
the  mission  environment 

• classes/ types  of  emitters  in  the  mission 
environment 

hostile  and  friendly  mix  (hostiles:  AAA  , 

SAM,  air  to  surface  mis sile  s , Ea,rly 
warning,  height  finders,  track-while- 
scan,  acquisition  radars , airborne 
interceptors,  GCA,  IFF  / SIF  and  telemetry 
signals , navigational  signals)  , communica- 
tions signals  (e.g.  , modulation,  GW,  amplitude 
frequency,  single  side  band,  etc.) 

maneuverable  emitters  (type;  heading,  altitude 
and  speed  envelopes) 

• number  of  emitters  displayed  simultaneously/  ;r 
instantaneously  (number  that  the  trainee  is 
capable  of  processing  in  his  span  of  control) 

• target  appearance/ disappearance/ attenuation  as 

a function  of  range  and  bearing  and  environmental 
conditions 


2.  EMITTER  RETURNS  (fidelity  dimensions 

• signal  signatures  per  class  of  emitter 


>-.■  rr 
.i.  iv.  * 
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5LE  6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 

« signal  jamming  (different  forms  of  voluntary 
jamming,  e . g . , broad  band,  spot,  fast  swept 
frequency  jamming,  modulated  jamming, 
transmission  deception) 

. emitter  distortions 

interference 

- simultaneity  of  signals  (overlap  of  signals) 

„ signal  characteristics  ( shape  , patterning  of  signals 
as  a function  of  range) 

m audio  characteristics 

multiple  audio 

noise 

• target  reactions  to  evasive  maneuvers  of  aircraft 

loss  and  re  acquisition 
deception  due  to  expendables 

• indication  of  weapon  effect  (hit/ miss  of  enemy  missile) 

. indication  of  jamming  (e.  g.  , in  cockpit) 

D.  VISUAL  REAL,  WORLD  (fidelity  dimensions  (2)@(4)  ) 

1.  TARGET  CHARACTERISTICS 

. size,  shape,  aspect,  silhouette  (configuration) 

« color,  visibility  -- v. 

, size  changes  as  function  of  range  . 

. movement/ movement  patterns  \ / 1 


. clutter 


NAVTRADEVCEN  69 -C -0298-1 


NAVTRADEVCEN  69  -C  -0298  - 1 


TABLE  6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 


2.  RANGE 


(fidelity  dimension  (^2)  ) 


• envelope  of  detection  ranges  achieved 

operationally  (appearance/ disappearance/ attenuation 
of  targets) 


3.  NON -TARGET  RETURNS 
(Classification  Cues) 

• reverberations 

• bottom  returns 


natural  (fish,  whale, 
reef,  pinnacle) 


obstructions  (wrecks) 


s torpedo  hydrophone  I 

effects  ( submarine)  j 

• cannisters , jammers , I 

decoys  (submarine  | 

launched) 

4.  BACKGROUND  CLUTTER/ 
NOISE  ON  DISPLAY 


' 5.  INTERFERENCE  EFFECTS  (fidelity  dimensions  ) 

FROM  MULTI-SHIP 
ENVIRONMENTS 

B.  RADAR 


(fidelity  dimensions  ) 


appropriate  video  and  audio 
characteristics 


Video  - appropriate  blip  size, 
shape,  brightness  for  the  bright- 
ness mode 

Audio  - appropriate  doppler 
effects,  echo  quality- 

appropriate  noise  spoke, 
appropriate  audio  characteristics 

appropriate  video  and  audio 
characteristics 


(fidelity  dimensions  ( 1 ) (3  ) ) 


1.  ENVIRONMENT 

0 wind  direction/ speed 
. sea  state 
: . V::  >:  . alti  tude 


ity  dimensions  (L)C2)) 
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TABLE  6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 


/ 


• land  mass/ elevations 

• artifacts  (stations,  buildings,  etc.) 

2.  RANGE  (fidelity  dimension  (?)  ) 

. envelope  of  detection  ranges  achieved 

operationally  (appearance/disappearance/ attenuation 
of  targets) 


3.  NON -TARGET  RETURNS 
. second  sweep  echoes 
„ radar  decoys 
e sea  returns 

_ cloud  cover 


, land  mass 

characteristics 

4;  BACKGROUND  CLUTTER/ 
NOISE  ON  DISPLAY 

5.  INTERFERENCE  EFFECTS 
FROM  MULTI-SHIP 
ENVIRONMENTS 

C.  ELECTRONIC  WARFARE 

1 . ENVIRONMEN  T 

• wind  dir ection/ speed 
„ altitude 


(fidelity  dimensions  (?)(?)) 

’'lip  characteristics  of 
operational  environment 


density  and  brightness 
correlated  with  sea  state 
and  range 


(fidelity  dimensions  (?)(?)  ) 
(fidelity  dimensions  (?)(?)) 


(fidelity  dimensions  ^(^  ) 


, weather  (temperature,  pressure,  humidity) 


i i 
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TABLE  6.  CLASSIFICATION  OF  SIMULATION  ELEMENTS  (Continued) 


2.  RANGE  (fidelity  dimension  ) 

• envelope  of  detection  and  lock-on  ranges 

achieved  operationally  (appearance /disappearance /attenuation 
of  targets) 

3.  NON-EMITTER  RETURNS  (fidelity  dimensions  (T)©  ) 

4.  BACKGROUND  NOISE/  (fidelity  dimensions  (T)©  ) 

INTERMITTENCY  ON 

DISPLAYS;  AUDIO 
INTERFERENCE 

5.  INTERFERENCE  EFFECTS  (fidelity  dimensions  <3XD> 

(EMI’ 

D.  GEOMETRY  OF  THE  (fidelity  dimensions  ©@©  ) 

ENGAGEMENT 

. Tactical  environment  (initial  conditions  and 
enroute  operations) 

• counteractions  and  weaponry  of  adversary 

characteristics  of  adversary  vehicles 

display  of  enemy  actions  (deployments, 
weapon  usage) 

display  of  enemy  weapons  action  (e.g.  , 
water  splash  of  surface  launched  air-to- 
water  missile) 

• adversary  tactic (intelligence  information, 

order  of  battle',  • 

E.  SCALE-  OF  THE  ENGAGEMENT  (fidelity  dimension  (2)  ) 

• ocean  area 

i 

• air  area 

/ .... 
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# Special  features  in  trainee  compartment  design  which 
presumably  will  enhance  training  value  in  terms  of 
greater  transfer  of  training  to  the  operational  syst  m. 

2.8  PHASE  5B:  THE  MANAGEMENT  OF  TRAINING- -STRUC  TURING, 

CONTROLLING  AND  MONITORING  TRAINING  AT  THE  INSTRUCTOR 
STATION.  In  addition  to  the  design  issues  involved  in  installing  the  environ 
ment  in  which  learning  can  take  place  (simulation  of  the  trainee  compart- 
ment), there  are  design  issues  associated  with  the  structuring,  monitoring, 
evaluation  and  control  of  training.  This  involves  the  management  of  train- 
ing at  the  instructor  station,  and  concerns  how  the  device  will  be  utilized 
as  a training  tool. 

The  human  factors  design  of  the  instructor  station  complex  empha-- 
sizes  the  concept  of  design  for  training  utilization.  Given  the  environment 
for  learning  (trainee  compartment  design),  the  concern  is  for  how  the  syn- 
thetic environment  will  be  used  for  training  and  for  creating  and  monitoring 
training  situations  in  order  to  shape  student  behaviors  and  to  optimize  the 
training  strategies  for  students  during  the  course  of  instruction,  The 
design  requirement  is  to  provide  a capability  for  structuring  training  so 
that  the  important  mission  and  system  events  can  be  installed  and  controls 
provided  to  insure  that  these  events  occur  in  prescribed  ways  at  prescribed 
times.  This  calls  for  a flexibility  in  training  capability  so  that  the  defined 
situations  can  occur  at  the  desired  times  in  order  to  satisfy  the  training 
objective s . 

A human  factors  design  pathway  is  developed  to  provide  an  instruc- 
tional capability  consistent  with  the  training  purpose  and  the  design  of  the 
trainee  compartment.  The  approach  consists  of  evaluating  the  information 
requirements  for  the  instructor  station  and  examining  the  design  alterna- 
tives, then  correlating  the  two  in  order  to  select  the  design  options  most 
relevant  to  achieving  the  desired  training.  Defining  the  design  pathway 
involves  evaluation  within  the  following  areas: 

a.  The  training  system  modes  of  operation 

b.  Analysis  of  instructor  functions 

c.  Instructor  requirements 

d.  Display  requirements 

e.  Control  requirements 

f . Monitoring  and  control  of  training  (enroute  during  the  exercise) 

g.  Fre-mission  requirements 

h.  Post-mission  requirements 

i.  Measurement  system  requirements 

j.  Communications  requirements 

k.  Overall  layout  and  workplace  requirements 


X 

A 
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2.8.1  Training  System  Modes  of  Operation.  The  initial  consideration 
involves  specifying  the  operating  modes  at  the  instructor  console  in  con- 
trolling, monitoring  and  evaluating  training,  and  in  the  instructing  of 
students.  These  modes,  which  range  from  strictly  manual  operation 
through  completely  automated  operation,  vary  considerably  in  required 
hardware  and  in  computer  programing,  and  hence  must  be  carefully  speci- 
fied at  the  outset. 

\ V • ■'  = 

Two  concepts  that  are  recognized  as  possessing  substantial  im- 
portance to  instructor  station  design  are,  the  full  automation  of  instructor 
functions  and  the  generalized  (modular)  construction  of  the  instructor  station 
The  desirability  of  these  two  design  concepts  is  tacitly  assumed  throughout 
this  phase  on  instructor  station  design.  Although  the  state-of-the-art  has 
not  advanced  sufficiently  to  suggest  these  as  ordinary  design  options,  they 
nevertheless,  represent  important  goals  to  be  achieved  as  data  and  tech- 
nique are  forthcoming  from  research  and  application.  Research  on  the 
automation  of  instructor  functions  is  described  in  Section  III  of  this  report. 

2.8.  1.1  Manual  Mode.  Decisions  must  be  made  about  instructor  involve- 
ment (controls  and  displays)  in  pre-exercise  briefing,  in  setting  up  the  in- 
itial conditions  for  any  training  exercise,  in  the  control  of  mission  events 
via  a script  of  event  setups  during  the  running  of  the  mission  scenario,  and 
in  post-exercise  critique.  Means  for  evaluating  trainee  performance  must 
also  be  specified,  ranging  from  subjective  judgments  about  performance  via 
displayed  information  and  trainee  actions  and  communications,  to  recorded 
measurement  information.  (Requirements  for  a measurement  system  are 
discussed  subsequently  in  Section  2,8.9). 

2.8. 1.2  Automated  Mode.  The  selection  of  instructor  functions  to  auto- 
mate involves  the  following  considerations. 

• Preprogramed  mission  scenarios 

An  important  feature  in  design  is  the  use  of  automated, 
standardized  preprogramed  training  exercises  with  the 
provision  for  instructor  intervention  in  prescribed  ways 
in  the  scenario  to  control  the  training  process.  To  be 
considered  are  control  and  display  capabilities  per- 
mitting scenario  alteration  at  certain,  times  during  a 
mission  (as  a function  of  stage  of  training)  in  order  to 
develop  a strategy  for  training  each  assigned  student 
to  achieve  the  training  objectives  for  a sequence  or 
course  of  instruction.  In  every  case,  the  prepro- 
gramed mission  scenarios  should  provide  a series  of 
exercises  graduated  in  difficulty.. 
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• Incorporation  of  automatic  assists 

This  involves  design  to  relieve  the  instructor  of 
the  more  "clerical’1  duties  in  order  to  enhance 
systematic  instruction.  It  is  based  largely  on 
automated  student-performance  monitoring  and 
evaluation  involving  an  automated  scoring  system 
with  computer  generation  and  display  of  scoring 
information.  More  precise  structure  and  control 
of  training  is  afforded  the  instructor  by  means  of 
automated  measurement. 

Again,  the  issue  of  instructor  intervention  must  be  considered  carefully 
since  it  is  highly  desirable  for  the  instructor  to  exercise  options,  via 
hardware  based  on  his  judgments  supported  by  displayed  and/or  recorded 
performance  information  on  how. well  each  student  is  doing.  Under  most 
conditions,  finer  control  of  training  is  afforded  in  this  way  rather  than 
in  purely  manual  or  in  fully  automated  control.  These  options  (described 
later)  enable  the  instructor  to  keep  each  student  at  "best  level"  of  perform-  v 
ance  throughout  the  training  sequence. 

2.8.  1.3  Adaptive  Training  Mode.  A decision  is  also  made  on  the  extent 
of  adaptive  training  desired  when  a partially  automated  (or  eventually  a fully 
automated)  training  system  is  selected.  Two  basic  approaches  to  adaptive 
sequencing  should  be  considered,  particularly  when  two  or  more  trainees 
are  performing  at  once  in  a simulator  but  independently:  a manual  (normal) 

adaptive  capability  or  an  automated  adaptive  capability. 

2.8.  1.3.  1 The  manual  (normal)  adaptive  training  capability  is  defined 

by  the  options  available  to  the  instructor  for  keeping  a student  progressing 
through  a sequence  of  training  at  his  own  achievement  speed.  Of  course, 
this  term  really  represents  the  ideal  expression  of  training  wherein  a 
skilled  instructor  does  his  job  to  train  a student  efficiently.  As  such,  it 
covers  most  instructional  situations  which,  of  themselves , are  not  depend- 
ent upon  equipment  assists.  However,  we  are  applying  this  term  particularly 
in  situations  where  automated  assists  will  increase  the  instructor  options 
in  developing  strategies  to  enhance  the  performances  of  a group  of  students 
undergoing  simultaneous  but  independent  training  in  a simulator.  These 
options  account  for  both  the  weak  and  for  the  exceptional  student  (individual 
differences  among  trainees) . As  the  student  progressively  acquires  skills  , 
the  task  and  mission  requirements  can  be  made  increasingly  more  difficult. 
This  emphasizes  that  the  instructor,  provided  with  the  necessary  student 
performance  information,  decides  on  the  strategy  and  initiates  the  sequenc- 
ing of  training  to  accommodate  each  student.  The  instructor  is  continually 
involved  throughout  the  mission  in  monitoring  all  critical  aspects  of  student 
performance.  He  evaluates  the  performance  of  each  student  relative  to  the 
mission  scenario  requirements  , being  concerned  with  those  who  make  errors 
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in  excess  of  preset  performance  tolerances  and  also  with  those  who  exceed 
scenario  performance  requirements.  He  provides  the  remediation  neces- 
sary to  correct  or  modify  deficiencies,  or  advances  the  difficulty  level  for 
a student  so  that  each  student  progresses  at  an  optimum  learning  rate.  A 
number  of  equipment  assists  are  required  to  provide  flexibility  in  employ- 
ing the  simulator  for  training  and  for  unburdening  the  instructor  . The 
features  which  enable  the  instructor  to  concentrate  bn  his  instructional 
duties  and  develop  training  strategies  tailored  to  the  individual  students 
include:  ' 


• problem  halt  or  manipulation  of  position  in  the 
mission  profile  (going  back  to  an  earlier  segment) 
for  remedial,  demonstration,  or  reinstruction 
purposes  ' 

• override  of  defined  programed  mission  events 

• automated  scoring  of  student  error  deviations 
from  a tolerance  level 

s automatic  halt  of  problem  when  student  exceeds 
a defined  error  envelope 

• on-line  modification  of  error  tolerance  envelopes 

. computer  generated  CRT  displays  of  student 

performance  information  in  all  relevant  instruc- 
tional modes 

• insertion  of  events  in  addition  to  the  parameters 
in  the  preprogramed  scenario  to  instruct  students 
whose  performance  exceeds  scenario  requirements  s 

< ■ .••••"••  • ' ■ 

• guidance  information  (cues  , prompts  , knowledge 

of  results)  provided  the  student  via  equipment 

. hard  copy  records  of  student  performance. 

Automated  evaluation  and  scoring  with  scoring  criteria 
adjusted  to  the  stage  of  training  of  each  student  provides  error  indications 
and  information  which  are  displayed  to  the  instructor.  He  uses  these  di- 
rectly in  monitoring,  evaluating  and  controlling  each  student's  progress. 
The  instructor  options  for  achieving  the  manual  adaptive  capability  (i.e.  , 
developing  the  training  strategies)  are  listed  below.  These  changes  can 
be  made  to  the  preprogramed  mission  either  by  event  or  time  access  via 
display. 3 and  controls  on  the  instructor  console. 


ERLC 
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• Continue  preprogramed  mission,  no  action  required. 

• Provide  verbal  feedback  of  performance  information 
to  selected  student  based  on  monitored  information 
obtained  from  the  displays  . This  may  be  in  the  form 
of  knowledge  of  results  of  performance  and/or  cues 
to  the  student  for  enhancing  performance. 

• Halt  the  problem  for  any  or  all  students  and  provide 
verbal  guidance , or  halt  the  problem  until  all  students 
have  completed  a mission  phase;  resume  mission. 

• Demonstration  mode  - -provide  a capability  during  the 
exercise  to  demonstrate  aspects  of  a task  or  maneuver 
to  all  students  or  to  demonstrate  specific  character- 
istics of  performance  for  any  or  all  students  when 
performance  is  below  expectations  for  the  specific 
mission  number;  resume  mission. 

• Reinstruction  mode- -provide  a capability  to  return 
to  an  earlier  portion  of  the  mission  for  any  or  all 
students  when  performance  is  below  expectation 
for  the  mission.  The  reinstruction  requires  that 
the  student  perform  again  that  which  he  has  just 
completed,  i.e.  , a segment  or  leg  or  portion  of 

a leg. 

• Error  alert  mode — when  a student  exceeds  the  error 
envelope  in  the  mission  for  a class  of  error,  a dis- 
play of  this  information  is  provided  the  instructor  and 
a computer -initiated  problem  freeze  is  a conventional 
outcome.  The  instructor  options  include;  manual 
override  of  the  freeze;  accept  freeze  with  procedures 
to  continue  (including  the  setting  of  new  error  toler- 
ance envelopes);  as  well  as  those  listed  above  in 
order  to  develop  the  training  strategy  for  the  student 
in  question. 

• Insert  new  events  into  the  exe r c i s e — whe n a studentTs 
performance  exceeds  the  mission  scenario  require- 
ments, the  instructor  has  the  option  of  inserting 
new  events  in  addition  to  those  in  the  preprogramed 
scenario.  This  provides  for  a controlled  increase 

in  problem  difficulty  for  the  purpose  of  keeping  any 
student  at  the  threshold  of  his  ability  at  any  given 
time. 


O 
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2.8.  1.3.2  The  automated  adaptive  training  capability'  requires  greater 

reliance  on  software  whereby  the  computer  is  involved  in  performing  the 
instructing  functions,  with  the  instructor  serving  as  a system  monitor. 

The  technique  automates  an  important  instructor  function, 
that  of  problem  selection  (that  is,  progressing  through  a graduated  series 
of  exercises  to  achieve  the  optimum  training  strategy).  This  involves  the 
automation  of  two  other  functions:  scoring  of  performance  so  that  adaptive 

sequencing  is  possible;  and  providing  information  so  that  the  student  will 
know  where  to  direct  effort  at  reducing  his  error  rate.  In  describing  an 
adaptive  training  system,  three  fundamental  elements  must  be  specified: 

a.  Performance  measurement  (continuous/ 
repetitive  scoring  of  performance) 

b.  Adaptive  variables  (task  modification  to 
systematically  change  its  difficulty) 

c.  Adaptive  logic  (the  function  used  to  auto- 
matically adjust  task  difficulty  based  upon 
measured  performance) 

The  implementation  of  adaptive  training  requires  that  a number  of  problems 
be  addressed.  The  problems  relate  to  the  following  (these  are  discussed  in 
detail  in  Section  hi). 

a.  Selecting  Adaptive  Variables  . These  are  the  parameters 
which  may  be  varied  to  affect  task  difficulty.  The  following  classes  of 
variables  are  appropriate  . 

1)  Task  loading  (increases  in  event  loading,  time 
of  event  introduction,  insertion  of  events  based 
on  threat  or  other  priorities) 

2)  External  forcing  function  ( e . g . , turbulence  on 
aircraft  control,  via  frequency,  amplitude 
bandwidth  of  function) 

3)  Noise  (signal  degradation) 

4)  Simultaneity  of  signals  per  unit  time 

5)  Quickening  of  a manual  control  system  (aiding) 

6)  Effective  time  constant  ( Te) 

7)  Malfunctions 
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b.  Feedback.  The  students  must  be  provided  information 
about  progress  throughout  training,  both  immediately  in  terms  of  how  the 
result  of  the  current  performance  conforms  to  an  objective  standard  (aug- 
mented feedback  provided  immediately  as  a performance  output  accrues 
and  is  identifiable),  and  during  and  after  a training  exercise  in  the  form  of 
a critique  of  performance. 

c.  Measurement.  The  crucial  measures  must  be  identified 
for  automatic  scoring.  Since  performance  envelopes  can  be  objectively 
defined,  measurement  of  deviation  from  them  can  be  accomplished.  In  a 
digital  computer-controlled  system,  all  parameters  of  activity  are  already 
being  computed;  thus  , the  measurement  becomes  a comparison  of  actual 
with  desir ed/ideal  performance. 


d.  Adaptive  Logic.  In  adaptive  training,  problem  difficulty 
level  is  set  in  response  to  student  performance  as  reflected  in  some  form 
of  error  score.  The  problem  of  error  scoring  for  adaptive  purposes  (as 
opposed  to  scoring  for  purposes  of  student  feedback)  would  be  the  major 
problem  area  for  programing,  in  that  adequate  guidance  from  a real-world 
model  or  from  research  is  lacking.  Initially,  values  must  be  selected  on  a 
somewhat  arbitrary  basis  until  their  suitability  can  be  established  in  the 
actual  training  setting.  i 

2.8.2  Analysis  of  Instructor  Functions  . Once  the  decision  is'  made  de-f 
fining  the  mode(s)  of  training  device  operation,  an  identification  and  analysi 
of  instructor  functions  is  undertaken  for  the  activities  involved  in  pre- 
exercise setup,  enroute  exercise  operations,  and  post-exercise  operations. 
The  display,  control  and  communications  requirements  are  specified  for 
the  following  classes  of  operations  and  instructional  functions. 

• Initialize  system,  confidence  checks 

• Initialize  system  for  mission  operation 

<»  Selection/modification/ construction  of  new  scenarios  (off- 
line) (where  applicable) 


• Student  briefing  (including  demonstration  mode  , where 
applicable) 

• Setup  of  initial  conditions  of  exercise  (manually  or  via 

address  of  preselected  data  sets  when  preprogramed 
scenarios  are  not  employed)  ' 

• Monitor  and  control  of  student(s)  performance  (provisions 
for  the  following): 
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Exercise  start  (COMEX) 

Exercise  freeze 

Manual  override  of  preprogramed  mission/scenario 
(as  applicable) 

Demonstration  mode 

Error  alert  mode  (computer  indication  to  the  instructor 
that  student  performance  is  out-of-tolerance  due  to 
excessive  error) 

Insertion  of  new  events  into  standardized  preprogramed 
scenario  (as  applicable) 

Control  of  events  throughout  non-programed  mission 
(time  and  event  control  of  problem  manipula tables) 

(as  applicable) 

Problem  speed  control  (real  time,  slower  than,  or 
faster  than  real  time) 

- Communications  with  trainees;  and  with  device  operator, 
remote  instructor(s) , as  applicable 

- Instructional  assists  for  unburdening  the  instructor 
• Evaluation,  and  scoring  requirements 


Measurement  system 


Mission  termination 


• Post-mission  critique 


Equipments  for  critique  (e.g.,  projection  system; 
audio,  audio/ video  recording) 


Hard  copy  critique  printouts  (form/ content) 


Time  and  event  printout  of  the  training  mission 
(form/content  of  hard  copy  for  school  records) 


A functional  overview  of  the  instructional  involvement  in  the  train 


ing  system  should  be  undertaken  to  consolidate  the  thinking  on  the  design 
philosophy  and  to  aid  in  the  further  definition  of  design  requirements  for 
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the  instructor  station.  A most  useful  form  is  a functional  flow  diagram  of 
the  operating  requirements  on  the  instructional  staff  for  the  complete  train- 
ing mission  cycle'.  An  example  of  this  functional  flow  analysis  is  shown  in 
Figure  12  for  an  airborne  Electronic  Warfare  Trainer.  ^ 

The  display,  control  and  communications  requirements  for  the 
identified  instructor  functions  can  now  be  analyzed  in  the  detail  necessary 
to  achieve  the  desired  design  pathway  for  the  instructor  station.  Table  7 
provides  an  example  of  the  level  of  analysis  that  will  serve  the  purpose  for 
specifying  the  functional  characteristics  of  design.  The  example  is  for  the 
same  airborne  electronic  warfare  trainer  cited/above. 

/ 

2.8.3  Instructional  Staff  Requirements.  The  makeup  of  the  instructional 

staff  must  be  defined.  This  includes  the  number  of  instructors  (based  on 
workload  estimates)  , and  types  of  instructors  to  be  used,  with  their  qual- 
ifications and  prerequisites  for  assignment  to  the  training  program.  Dis- 
tinctions should  be  made  about  type  of,  and  assignment  of  instructors. 
Usually,  three  categories  can  be  Refined:  the  central  or  chief  instructor, 

located  at  the  master  console(s);  fihe  device  operator,  located  at  the  compu- 
ter complex,  vfrho  assists  in  operating  the  training  system;  and  remote 
instructors,  located  in  or  adjacent  to  specific  trainee  compartments  (or 
also  at  the  central  instructor  station)  to  monitor  performance  and  provide 
instructional  support  as  required. 

- • ^ 

An  important  consideration  is  the  specification  of  instructor  to 
student  ratios,  particularly  in  multi- student  situations  where  each  student 
performs  independent  of  the  others  . 

2.8.4  Display  Requirements.  A key  requirement  in  instructor  station 
design  is  the  efficient  display  of  student  performance  information.  The 
specification  of  display  requirements  is  largely  dependent  on  the  informa- 
tion requirements  in  terms  of,  content;  amount,  variety,  and  speed  of 
information  change;  complexity;  and  the  number  of  trainees  and  trainee 
stations  involved  (multi- trainee  stations  such  as  found  in  tactical  team 
trainers  or  independent  multi-cockpit  devices).  In  short,  an  organized, 
unambiguous,  fast-access  information  format  is  desired  commensurate 
with  the  device  under  consideration. 


^The  design  is  for  an  Air  Force  Airborne  Electronic  Warfare  training 
system  proposed  to  simulate  the  major  current  and  predicted  mission 
..environments.  It  exemplifies  a semi-automated  system  (preprogramed 
mission  scenarios  modifiable  at  the  instructor  station;  automated  moni- 
toring, evaluation  and  scoring)  . Four  independent  student  stations  can 
be  handled  during  a training  exercise  by  one  instructor. 
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Figure  120  Functional  Flow  Analysis  for  EW  Trainer 
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TABLE  7.  INSTRUCTOR  FUNCTION  ANALYSIS  (Continued) 

[ Display  Control  Communications 
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2.8.4.  1 What  Must  Be  Displayed.  The  information  classes  to  be  dis 
played  should  account  for  the  following: 

• Mission  areas — the  mis sion/  tactical  environment/ 
relative  geometry  in  the  pertinent  display  modes 

. Target  data  — 

target  c ontact- -( onset/ attenuation/  disappearance 
of  targets  as  a function  of  range  and  other  pertinent 
factors) 

tracks,  differentiation  of  target  classes  (e.g.,  EW 
emitter  types),  positions  in  the  environment 

range/bearing  information,  aspect,  depth/ altitude 

9 Own- ship  and  support  units  — 

identity,  position  and  tracks,  movement 

• Indication  of  the  system  state/  simulator  mode  of 
operation  and  the  present  phase  or  point  in  the 
mission 

• Status  of  events  (e.g.  , indication  of  jammer  activation, 
chaff  activation) 


• Performance  information  on  each  student  (error  indica- 
tions , scoring  information)  in  the  pertinent  display 
modes  and  formats 


Indication  of  the  student  control/ tactical  actions  and 
consequences  of  student  actions,  for  monitor  and  control 
of  training  (e.  g.  , range  scale  settings,  sector  selection, 
selection  of  operating  modes  such  as  MAD » deployment 
of sonobuoys) 

Indication  of  student  detec tiori/  identification  of  targets 
once  displayed  (event  and  time  of  event) 

Indication  of  instructor  actions  once  initiated/ controlled 
over  time 


Information  for  manual  control  functions  (e.g.  , position, 
heading,  altitude/ depth  and  speed  information  for  1 •flying 
an  aircraft  or  .controlling  a submarine  target) 


* m o 
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2. 8. 4. 2 Display  Techniques.  Decisions  on  primary  display  techniques 

(media  and  mechanization)  for  monitor,  evaluation,  and  control  of  training 
are  based  on  a correlation  of  the  information  requirements  for  the  system 
and  the  characteristics  of  classes  of  display.  The  choices  range  aicross 
the  following: 


• Repeater  displays  (reproductions  of  trainee  displays) 

. Event  on/off  indications  ( e . g . , indicators  depicting  an 
out- of-tole ranc e event,  the  trainee  mode  selection,  or 
instructor  insertion  of  an  event  { control/ display  inte- 
gration)) 

. [Plotting  boards  (mechanized) 

» Computer  generated  displays  — synthesized  multi  -format/ 
integrated  displays  (e.g..  Cathode  Ray  Tube  display  of 
alpha-numeric / pictorial- situational  information) 


2. 8. 4. 2.  1 Criteria  for  Defining  Primary  Display  Requirements  . De- 

cisions on  primary  display  techniques  and  display  modes /format  require- 
ments should  consider  the  following  criteria. 

• Information  handling  capability  (amount,  types, 
rapidity  of  change,  number  of  trainees  under 
control) 


. Multi-format  requirements,  high  density  informa- 
tion display,  information  integration  (alpha-numeric 
and  situational/ graphic  display  requirements  involving 
discrete  event  and  overall  situation  displays;  con- 
tinuous alpha-numeric  error  readouts) 

* Rapid  mode  changes  for  monitoring , control  and 
evaluation,  requirements  for  monitor  and  control 

- o f iner e than  one  trainee  sequentially 

* Display  flexibility  (with  relevant  controls) 

- mis s ion/ geographic  areas 

- scale  depictions 

- information  classes 

- display  modes  : i ^ T ; . 

- coding  of  displayed  information 

* Accuracy , reliability  and  registration  (information  . 
alignment)  of  the  visual  displays 
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. Expansion  capability  (fox  future  display  requirements) 

Visual  factors  (acuity,  luminance,  refresh,  rate/ flicker  , 
contrast  sensitivity  s resolution,  clutter) 

2.8 .4.2 . 2 Ancillary  Display  Requirements  . Requirements  are  also 
specified  for  those  displays  not  primarily  involved  in  the  instructional 
process  but  which  are  necessary  to  training  system  operation  (e.g.  , 
mission  timers,  system  status  displays,  etc.). 

2.8.5  Instructor  Console  Controls.  To  achieve  full  use  of  the  instructional 

capability,  a number  of  classes  of  controls  are  required  to  operate  the 
training  device  and  to  enable  the  shaping  of  behavior  in  the  trainee  com- 
partments). The  control  classes  include  the  following: 

• Device  operation — start,  halt,  reset,  override. 

• Means  for  selecting  displays  and  display  modes 
(multi-format  displays)  . 

• Means  for  controlling  vehicles  (e.g. *  * controls  for 

* ff  lying  * * an  aircraft,  steering  a target  submarine). 

• Means  for  positioning  units  in  the  mission,  both 
initially  at  problem  start  and  during  the  mission. 

« Means  for  inserting  and  manipulating  targets  (e.g., 
time  and  position  of  entering  targets,  repositioning 
of  targets,  control  of  range  between  targets  or 
between  own-vehicle  and  targets). 

• Means  for  direct  access  to  the  device  computation 
system  (manual  input  controls  such  as  keyboard,  light  pen)  . 

Means  for  inserting  new  error  tolerance  into  computer 
( & error)  when  trainee  exceeds  current  programed 
error  limits  . 

Environment  (media)  controls. 

Communications  controls. 


• Means  for  simulating  failures  / emergencie  s . 
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• Means  for  inserting  scenario/ mission  events  during  an 
exercise  for  both  non- programed  and  preprogramed 
exercis e s . 


• Means  for  eliminating  defined  programed  events  (for 
example,  in  preprogramed  scenarios,  '’erasing"  an 
airborne  interceptor  threat  in  an  EW  defensive  mission 
that  is  scheduled  to  appear  but  is  not  desired  at  this 
time  because  of  the  training  strategy  being  developed 
for  a particular  student). 


2.8.5.  1 Selection  of  Controls.  Decisions  on  the  selection  of  controls 
for  operating  the  instructor  station  and  controlling  the  training  are  based 
on  the  following  criteria: 

o Display-control  compatibility  (compatibility  with 
tabular  and  graphic  formats). 

• The  amount  and  types  of  information  and  speed 
requirements  in  information  reception  and 
transmission. 

• Number  and  size  required  to  avoid  clutter, 
ambiguity  and  needless  redundancy  because 
of  the  potential  for  error  in  usage. 

• Ease  of  operation,  particularly  where  in- 
structor to  student  ratios  are  high. 


• Efficiency  of  panel  space  usage. 


Instructor  console  controls  can  be  grouped  according  to  the 
functions  performed  (obtained  from  the  instructor  functions  analysis)  which 
in  turn  affects  the  choices  incontrol  selection. 


2.8. 5.  1.1  Mission/ Instructional  Controls  - -(on-line)  . These  are  the 

major  groups  of  controls  used  for  structuring  and  controlling  training  dur- 
ing the  exercise,  and  range  from  switches  through  control-display  indica- 
tors to  manual  input  devices  . In  multi-man  training  systems  where  fast 
access  to  much  information  in  multi- formats  is  necessary,  the  selection 
of  manual  input  controls  with  compatible  displays' is  an  important  decision 
for  training  management.  The  alternatives  available  in  the  selection  of 
controls  include  the  following: 


Switch/  indicator  ar  ray  ( engraved  legends) 


• Digit  keyset 
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* Alpha-numeric  keyboard 

. Joystick  (cursor  symbol  positioner) 

. Trackball  (cursor  symbol  positioner) 

« x-y  multi- gain  push  buttons  (cursor  symbol  positioners) 

* light  pen  (or  light  gun) 

2.8.5. 1.2  Scenario  Modification- -(off- line)  . These  are  the  controls  for 

modifying  or  generating  new  scenarios  when  preprogramed  training  exer- 
cises are  employed.  For  the  manual  mode,  the  pertinent  controls  are  those 
for  the  efficient  setup  of  the  initial  conditions  for  an  exercise. 

2.8.5.  1.3  Ancillary  and  Power  Supply.  These  are  the  controls  associ- 

ated with  the  operation  of  the  training  device,  e.g.  , power  to  instructor 
console,  mission  initiation  (control  for  starting  the  mission  scenario), 
mission  program  generation  (control  for  initiation  (off-line)  of  a new  mission 
e.g.,  to  call  up  a page  of  CRT  information  in  order  to  select  desired  param- 
eters for  a new  mission  scenario),  emergency  stop,  lamp  test,  etc. 

2 . 8 . 5 . 1 4 Communications  . These  are  standard  controls  , most  often, 

replicas  of  existing  operational  equipment  for  two-way  communications 
with  a student(s)  (individual  or  group);  with  a remote  instr uc tor ( s) ; with  a 
device  operator  and  other  associated  personnel. 

2.8.6  Monitor  and  Control  of  Training  (during  the  training  exercise).  To 
achieve  the  training  purpose  and  the  training  objectives  for  the  system,  the 
characteristics  of  the  instructional  capability  must  also  be  specified.  The 
instructor  must  be  provided  displays  of  necessary  performance  information 
on  each  student  and  the  controls  and  communications  necessary  to  develop  a 
training  strategy  for  each  student.  The  options  to  provide  the  instructor 
(via  hardware)  for  enhancing  the  training  value  of  the  device  include  the 
following: 

m Provide  monitoring  and  control  of  performance  via  error 
displays  at  the  console.  This  information  is  used  in 
judgments  for  continuing  the  mission  as  preprogramed 
(automated  scenarios)  or  as  planned  (manual  scenario 
setup);  or  for  remedial  action  (changes  to  preprogramed 
mission  or  changes  in  event  insertion  via  script)  . Verbal 
communication  capability  with  trainee  is  required  through- 
out the  training  exercise.  An  example  of  a student  per- 
formance information  display  is  provided  in  Figure;  13.. 

This  is  a sample  of  a basic  CRT  page  format  showing  a 
recommended  positioning  of  the  displayed  information. 


Status  information--student  number;  viewing  mode 
(monitor,  demonstration,  etc.) 
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Figure  13.  Basic  Layout  and  Placement  of  Information  for  a CRT 
Page  Format  (Alpha-numeric). 
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In  this  case,  the  instructor,  by  means  of  a light  pen 
control,  is  able  to  call  up  various  classes  of  informa- 
tion (alpha-numeric)  on  a given  student.  A sample  of 
a CRT  page  format  of  alpha-numeric  information  is 
shown  in  Figure  14.  A situational  (pictorial)  display 
format  is  also  desirable  (see  Figure  15).  The  infor- 
mation in  these  figures  refers/ to  electronic  warfare 
simulation.  An  example  of  a Combined  graphic  and 
alpha-numeric  CRT  format  is  shown  in  Figure  16  for 
a proposed  multi-student  navigational  trainer  (Bark, 
et  al,  1969). 

Provide  demonstration  mode  capability  for  any  or  all 
students  during  briefing  and  also  enroute  in  the  exer- 
cise (problem  halt  or  with  problem  underway). 

Capability  for  problem  halt  and  recycling  of  a portion 
of  the  mission  to  reinstruct  an  erring  student  in  a 
defined  portion  just  completed. 

Capability,  in  preprogramed  scenarios,  to  manually 
override  any  event  entry  when  it  is  in  position  for 
entering  the  mission. 

Provide  , in  preprogramed  scenarios,  options  to 
advance  individual  students  at  their  best  rate  of 
progression  (i.e.  , based  on  an  automated  monitoring 
and  scoring  capability).  This  can  be  accomplished 
for  those  who  have  demonstrated  performance  ex- 
ceeding scenario  requirements  for  a specific  mission 
or  sequence  of  training.  Additional  events  and  mal- 
functions (in  type  and  frequency)  can  be  inserted. 

Provide  a capability  to  call  up  records  of  automatically 
scored  student  performance  for  the  purpose  of  critique, 
either  on-the-spot  during  a mission  or  after  mission 
completion,  and  for  detailed  hard  copy  of  performance 
for  school  record  keeping. 

Provide  problem  halt  capability  for  any  or  all  students 
for  demonstration,  guidance  or  administrative  hold. 

Provide  for  control  of  the  speed  of  the  training  scenario 
problem  (slow/  r eal/fast  time)1. 


Monitoring  Student  1 Pre-Flight 


Figure  14.  CRT  Page  Format  of  Alpha-numeric  Information 

(the  example  is  for  error  analysis  in  pre-flight  setup). 
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Figure  15.  CRT  Situational  Display  Format  (Electronic 
Warfare  Mission  Scenario). 


Cfiw  designation 
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2.8.7  Pre-mission  Requirements.  The  hardware  implications  in  instruc- 
tor pre-mission  activities  involve  the  following; 

• System  confidence  checks. 

• Setup  of  initial  conditions  for  an  exercise  in  non- 
automated  systems  (i.e.  , manual  insertion  by  the 
instructor). 


• Capability  for  the  selection  of  preprogramed  mission 
scenarios  in  automated  systems  , also,  the  capability 
(off-line)  to  modify  an  existing  scenario  or  generate 
an  entirely  new  scenario  as  training  objectives  are 
modified  or  new  training  requirements  emerge. 

• Capability  for  in- station  briefing,  as  required,  in- 
cluding demonstration  utilizing  trainee  compartment 
displays  . 

2.8.8  Post-mission  Requirements.  The  hardware  implications  in  instruc- 
tor post- miss  ion  activities  involve  the  following: 

• Capability  for  recycling  the  system,  when  automated 
preprogramed  scenarios  are  used,  or  for  shutting 
down  the  device. 


Capability  to  call  up  hard  cop^t  printouts  of  student 
performance  indications  for  (in-line  critique  (format 
and  content)  and  for  school  record  keeping  (format 
and  content  for  off-line  analysis). 

Hardware  for  providing  mission  critique  in  the  trainee 
station(s). 


Special  equipment  for  critique  required  in  the  team 
training  or  multiple  student  contexts  ( e . g . , visual 
projection  displays  , audio  and  audio/ video  recording  , 
etc.  V in  a central  critiquing  area  for  the  reconstruc- 
tion of  the  mission  just  accomplished). 

2.8.9  Measurement  System . Understanding  , modifying  and  predicting 
human  behavior  depend  upon  measurement  ope  rations  . Knowledge  about 
the  outputs  of  performance  provides  a basis  for  assessing  the  effects  of 
training  and  for  predicting  performance  under  future  conditions  . In  fact, 
^performance  measurement  is  the  bridge  between  the  trainee  outputs  and 
the  information  requirements  at  the  instructor  station.  Thus , measure- 


: -ment  requirements  involving  the  specification  of  what  and  how  to 
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the  software  programing,  and  the  utilization  of  display  and  recording  equip- 
ment must  be  carefully  evaluated. 

Assuming  the  decision  has  been  made  to  provide  a measurement 
of  performance  capability  in  the  device,  an  effort  is  undertaken  to  determine 
the  scoring  requirements  and  means  for  implementing  the  scoring  capability. 
The  human  factors  inputs  in  developing  the  measurement  capability  will  be 
described  here  for  the  most  demanding  case,  that  of  an  automated  measure- 
ment system  involving  computer  monitoring,  scoring  and  evaluation.  For 
training  devices  in  which  automatic  scoring  and  evaluation,  and  data  record- 
ing are  not  warranted,  the  inputs  can  be  selected  as  needed  from  the  follow- 
ing discussion. 

The  development  of  any  proficiency  measurement  system  is  a 
difficult  undertaking  involving  technically  complex  issues.  A number  of 
requirements  must  be  satisfied,  and  measurement  errors  accrue  that  are 
directly  attributable  to  failures  in  satisfying  these  requirements  . The 
more  prominent  among  these  relate  to  reliability,  validity,  behavioral 
sampling  practices,  the  selection  of  measures,  the  characteristics  of  the 
measures  (objective  vs.  subjective,  overall  vs  . diagnostic,  individual  vs. 
team),  levels  of  measurement  ( i . e . , types  of  measurement  scales)  and 
mathematical  and  statistical  operations.  These  issues  have  been  examined 
in  depth  in  a number  of  substantial  documents  . It  is  sufficient  for  our 
purposes  to  call  the  reader’s  attention  to  these.  Representative  sources 
from  this  large  store  include  the  work  of:  Glaser  and  Klaus  (1962); 

Lindquist  (1951);  Buckhout  and  Cotterman  (1963);  Smode,  Gruber  and  Ely 
(1962);  Ericksen  (19  52);  Krumm  and  Farina  (1962);  and  Greer,  Smith  and 
Hatfield  (1962). 

Assuming  the  above  mentioned  c onsiderations  , the  development  of 
the  measurement  capability  for  a training  device  involves  the  accomplish- 
ment of  a series  of  logical  inter  related  steps  . This  sequence  of  operations 
is  described  next. 

2*8.9. 1 Identify  the  Important  and  Critical  Tasks.  Based  on  the  system 
and  job  analyses,  the  requirement  is  to  break  down  individual,  subteam, 
and  team  tasks  into  manageable  units  of  performance  that  are  observable 
and  quantifiable.  Decisions  must  be  made  in  selecting  more  discrete  units 
of  behavior  and  more  comprehensive  segments  of  performance . The  de- 
sired product  is  a list  of  things  worthy  of  measurement. 

An  outline  of  the  task  structure  is  the  necessary  basis  for  the 
development  of  measurement  classes  and  associated  scores  . What  is  needed 
here  has  alreadyjbeen  accomplished  in  Phase  3 and  it  is  mostly  a matter  of 
correlating  skills  and  knowledge  requirements  with  scores  that  are  rele- 
vant and  obtainable.  Additional  insights  may  be  gained  by  means  ranging 
from  discussions  with  operational  personnel  knowledgeable  in  the  type  of 
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training  contemplated  to  flow  charts  of  the  computer  operations  involved  in 
monitoring  student  activity. 


Using  the  electronic  warfare  training  system  example  cited 
earlier,  the  initial  cut  at  task  structure  needed  for  organizing  the  measure- 
ment requirements  can  be  described  as  follows: 

• Procedures  following--heavy  emphasis  on  switch 
positioning  throughout  the  mission  scenario  (setting 
up  equipments  , configuring  equipments  to  obtain 
desired  parameters  , making  mode  changes  , select- 
ing bands  , peaking/optimizing  signals,  etc.) 

. Signal  detection--( visual,  aural) 

. Signal  recognition  and  pattern  comparison- -display 
interpretation  and  recognition  of  signal  signatures 

. Decision  making- -decision  options  within  the  bounds 

of  EW  doctrine,  threat  assessment  in  terms  of  priority 
search  and  threat  priority 

• Communications 

• Log  keeping 

. Crew  coordination 


2. 8.9. 2 Quantify  the  Performance  Elements  and  Select  Measures 
Appropriate  to  the  Behavior  to  be  Evaluated.  Quantifying  behavior  involves 
the  consideration  of  the  missions  performed  in  the  system  and  the  selection 
of  parameters  which  are  useful  and  can  be  obtained  efficiently . A variety 
of  measures  are  available  for  describing  specific  behavioral  sequences  in 
performance.  The  selection  of  measures  pertinent  to  the  purpose  of  train- 
ing should  include  both  diagnostic  indicants  of  performance  as  well  as  sys- 
tem output  measures  . •<"•••••' 


As  an  aid  to  planning,  an  organization  of  basic  types  of  measures 
that  may  be  used  in  combination  is  provided  in  Table  8 . These  classes  of 
measures  are  graded  on  a quantitative-qualitative  continuum  with  precise 


quantities  (time,  accuracy,  frequency)  at  one  end  and  more  qualitative 
interpretations  at  the  other  end.  Each  class  includes  a variety  of  subgroups 
and  specific  measures  (Smode,  Gruber  and  Ely,  1962;  SAE , 1968). 


The  selection  of  measures  for 
reflect  the  following \ 


the  measurement  system  must 


NAVTRADEVCEN  69-C-0298-1 


. Both  objective  and  subjective  measures  are  desired. 

There  are  performances  that  do  not  lend  themselves 
to  quantification  or  objective  scoring.  In  fact,  larger, 
relatively  richer  segments  of  performance  are  best 
assessed  by  direct  observation  (e.g.  , team  coordination). 

. Both  diagnostic  measures  (indicants  of  weakness,  strength 
in  skills  and  specific  performances  within  the  general  capa- 
bility)  and  system-terminal  measures  (indicants  of  system 
output)  are  required. 


• Measures  must  consider  the  interactive  effects  of  man 
and  equipment.  The  measurement  requirements  are 
highly  specific  to  each  system  in  terms  of  mission  re- 
quirements and  the  equipments  operated.  The  classifi- 
cation of  measures  in  Table  8 provides  assistance  in 
identifying  the  types  of  measures  meaningful  in  any 
system  context.  The  measures  selected  need  then  to 
be  put  in  terms  specific  to  the  contemplated  training 
system . 

~ . The  use  of  measures  describing  infrequently  occurring 

events  is  desirable.  Performance  on  improbable  events 
(involving  safety/hazard,  emergencies  and  malfunctions) 
provides  data  of  value  for  training. 

Two  specific  examples  are  provided  concerning  the  selection  of 
appropriate  measures.  The  first  example  concerns  the  airborne  EW  system 
cited  earlier.  For  the  basic  EW  defensive  mission,  the  computer  scorable 
measurement  classes  and  applicable  scores  are  organized  within  segments 
of  flight  in  the  mission  cycle.  The  measurement  classes  concern  the 
following: 


• Switch  setup  positions 

. Equipment  preset  (bands/frequencies  requirements) 
. Switch  setup  for  the  signal  analysis 
. Threat  assessment 
. Switch  setup  for  jamming 
. Initiation  of  jamming 
. Use  of  expendables 
. Threat  priority 
. Environment  coverage 
. Evasive  maneuvers  : : 

. Communications 
. Switch  shutdown  positions 
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TABLE  8.  A CLASSIFICATION  OF  MEASURES 


2 . 


3. 


4. 


TIME:  Measures  dealing  with  time  periods  in  production  of  performance. 

1.  Time  to  Initiate  an  Activity  from  the  Onset  of  a Signal  or  Related 
Events 

Time  to  perceive  event 
Reaction  time 

Time  to  initiate  a correction 

Time  to  initiate  a subsequent  activity  (following  completion  of 
a prior  activity) 

Time  to  initiate  a course  of  action 
Time  to  detect  trend  of  multiple  related  events 
Time  to  Complete  an  Initiated  Activity 
^ Time  to  acquire,  to  lock-on,  to  identify 
Time  to  complete  single  message 
Time  to  complete  a computational  problem 

Time  to  make  an  adjustment/  manipulation/  control  positioning 
Time  to  reach  a criterion 


Overall  Time  from  Signal  Onset  to  Activity  Completion 
Percent  time -on -tar get 

Time  spent  in  an  activity  (communicating,  repairing,  comput 
ing,  etc.)  ' i ; 

Time  to  complete  a sequence  of  activities 
Build-up  of  time  (cue  length).;w-.> %;••••/• 

Distribution  of  Part  Task  Times  in  Completing  an  Activity 
Tiirie'--  sharirig'ampng;i  yer^tST.  1 '?  '• 
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— CLASSIFICATI0N  OF  MEASURES  (Continued) 

ACCURACY:  Measures  dealing  with  th«=> 

production  of  ^ 

Correctness  of  Observation  or  Perception  (Discret e/seque^ 
Accuracy  in  identifying  display  readout 

ground  ^e^raiii^cele^tia^nav^gation^objec^')15  (environm^t ■ 

Accuracy  in  estimating  distance,  direction,  speed 
Time  estimating  accuracy 

Detection  of  a trend  based  on  multiple  related  events 

Detection  of  change  in  presence  of  noise 

Correctness  of  observation  sequence 
^ * Correctness  of  Response  or 


Accuracy  in  control  positioning  ( pressures  d™ 

tude  rate,  and  duration)  \pressures,  direction,  ampH- 


Accuracy  of  in-flight  maneuvers 
Accuracy  of  retrofire  maneuvers 
Accuracy  of  intercept 
Computing  accuracy 

Selection  of  action  from  among  alternatives 
Correct  symbol  usage 
Accuracy  in  s patial  pos i 

Accuracy  in  weapon  delivery 

i Accuracy  i n , ; 
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TABLE  8.  A CLASSIFICATION  OF  MEASURES  (Continued) 


3.  Error  Magnitude 

Error  amplitude  measures 
Error  frequency  measures 
Error  distance  (circular  or  linear) 

4 . Correctness  of  Respons  e Sequence 

Sequence  of  response 

Sequential- manipulative  accuracy  ( serial  response  , one  acti- 
vity^ coordinated  response  with,  several  controls) 

5.  Adequacy  of  Probability  Estimation  (Relative  to  an  ’Ideal  Observer1) 


Accuracy  in  using  unreliable  information 

Recognition  of  out- of- tolerance  condition 

FREQUENCY  OF  OCCURRENCE:  Measures  dealing  with  the  rate  of 

repetition  of  behavior  . 

1 . Number  of  Responses  Per  Activity  or  Interval 
Number  of  actions  made  per  unit 

Number  of  communications  per  activity  or  interval 

Number  of  adjustments  to  maintain  in -tolerance  (number  of 
checks,  replacements,  problems  solved) 

Number  of  interactions  with  other  members 


Number  of  gross  / significant 
2 . Number  of  Defined  Consequences 


errors  per  unit 
of  Performance  Per  Activity 


Number  of  out-of-tolerance  conditions 


O' 
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TABLE  8.  A CLASSIFICATION  OF  MEASURES  (Continued) 

3 . Number  of  Observing  or  Data -Gathering  Responses 

Number  of  requests  for  information 

Number  of  inter  rogations  / observations  made 

Number  of  discrete  recordings  / reportings  made 

AMOUNT  ACHIEVED  OR  ACCOMPLISHED:  Measures  dealing  with  the 

amount  of  output  or  accom- 
plishment in  performance. 

1.  Response  Magnitude  or  Quantity  Achieved 

Degree  or  proportion,  of  success  (intercepts,  information 
collection,  weapon  delivery,  rescue,  landing,  etc.) 

Cumulative  response  output 

Written  test  of  knowledge  (scores) 

2.  Man-Machine  System  Achievement 

Attainment  of  training  objectives 

Assessment  of  "merit”  in  performance  (influenced  by  man- 
machine  interactions) 

CONSUMPTION  OR  QUANTITY  USED:  Measures  dealing  with  resources 

expended  in  performance  in  terms 
of  standard  references  . ; 

1.  Resources  Consumed  Per  Activity 

Fuel-energy  conservation 

Units  consumed  in  activity  accomplishment 
Y 2.  Resources  Consumed  Per  Time  u " 

Rate  of  consumption 
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TABLE 


A CLASSIFICATION  OF  MEASURES  (Continued) 


HAVIOR  CATEGORIZATION  BY  OBSERVERS:  Measures  dealing  with 

classifying  more  complete 
behaviors  into  opera- 
tionally defined  subjec- 
tive categories  . Obser- 
vations are  placed  into 
discrete  classes  on  a 
continuum  for  the  event 
observed. 

1 . Classifying  Activities  or  Handling  of  Events 

Impromptu  response  invention  (improvising) 

Communication  effectiveness 


Redundant  communications 
Emotional  content  of  communication 

Priority  assignment  to  an  activity  or  among  activities 
2.  Overall  Judgments  of  Performance 
Coordination  of  effort/ movement 
Procedural  synchronization  of  action 
Relevance  of  response 

Substantive  content  of  communication 

Intelligibility  of  voice  report  : 

Use  made  of  available  references  , job  information,  test 
equipment 

Visual-perceptual  orientation 

. ::  • V..;.  1 ' 

Crew  cohesiveness  . 

Quality  of  checks  (fault  location) 


checks  / errors. 


Use  made  of  performance  information  available  from 


me 
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TABLE  8.  A CLASSIFICATION  OF  MEASURES  (Continued) 
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Table  9 identifies  the  scores  obtainable  (via  computer  software 
programs)  for  the  defined  classes  of  measures.  Where  other  mission  re- 
quirements are  distinct  from  the  selected  mission  the  additional  measure- 
ment classes  are  incorporated  in  the  listing.  Thus,  in  the  same  EW  exam- 
ple, the  basic  reconnaissance  mission  (ELINT)  requires  measurement 
classes  in  addition  to  those  shown  for  the  defensive  mission.  These  include: 
signal  setup  for  DF  beaming,  Recording,  analysis  of  signal,  subsequent  DF 
bearings,  priority  search,  and  general  area  reconnaissance. 


The  second  example  involves  an  Operational  Flight  Trainer 
whe rein  measurement  is  concerned  with  flight  and  navigational  skills.  In 
this  instance,  computer  programing  provides  a,  record  (hard  copy  printout) 
of  selected  parameters  of  defined  maneuvers  when  any  given  parameter  ex- 
ceeds a preset  range  of  parameter  deviation  (the  instructor  is  able  to  pre- 
program the  tolerance  limits  for  each  parameter).  Various  types  of  meas- 
ures are  needed  to  describe  the  performances  selected  for  assessment. 
These  defined  measures  are  correlated  with  appropriate  flight  parameters, 
and  serve  to  assist  the  instructor  in  selecting  only  those  parameters  (for 
display)  which  have  diagnostic  relevance  to  the  performance  being  examined. 
Table  10  identifies  the  parameters  available  to  the  instructor  for  measuring 
defined  aspects  of  trainee  performance. 

2. 8. 9. 3 Establish  Performance  Criteria.  Standards  of  performance  are 

required  if  the  proficiency  measures  are  to  predict  on-the-job  performance. 
Standards  (predetermined  norms  defining  acceptable  tolerances  for  perfor- 
mance) provide  the  basis  for  the  comparison  and  evaluation  of  student 
performances  and  hence  relate  importantly  to  training  strategy. 


Various  means  are  available  for  achieving  these  criteria. 
Standards  can  be  set  for  a number  of  performances  from  the  rational  anal- 
ysis of  the  system  and  task  structure.  For  certain  behaviors,  adequacy  of 
performance  may  be  differentiated  on  the  basis  of  time  to  perform.  Re- 
action time  may  be  critical  or  error  response  may  be  detrimental  unless 
corrected  (assuming  adequate  feedback  of  performance  information)  within 
a short  time  period  after  the  error  response.  In  other  instances,  emphasis 
may  be  placed  on  quality  and  precision  of  response  or  on  the  integration  of 
activities  that  indicate  an  ability  to  cope  with  a range  of  unforeseen  con- 
tinge ncies  . 


Another  approach  to  the  development,  of  interim  standards  in- 
volves the  use  of  the  a priori  model  of  system  and  vehicle  performance 
which  is  developed  during  the  design  phases  of  the  operational  system.  The 
relationships  between  components  as  specified  in  the  model  can  at  times  be 
translated  into  terms  allowing  a quantitative  expression  of  performance 
levels  required  for  successful  performance  . 
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TABLE  9.  BASIC  EW  DEFENSIVE  MISSION  MEASURES  AND  SCORES 


Mission  Phase  Measure  Scores 


Flight  planning  N/A  N/A 

Pre-flight  setup  Switch  setup  positions  Sequence  of  switch  positions 

(checklist  format) 

Switch  setting  error 

Omission  of  switch  setting 

Malfunction  correction 

Communications  (oosition  of 
channel  switches) 

Equipment  preset 
(bands/frequencies 
" * requirements) 

Switch  positions  in  equipment 
preset 

Emitter  Switch  setup  in 

environment  signal  analysis 

Switch  position  in  detection 
(signal  to  be  countered) 

Time  to  detect 

Failure  to  detect 

• - Cue  buildup  (onset  of  signal 

. \ to  action) 

Switch  position  in  identifica- 
tion (wrong  emitter) 

Time  to  identify  type  of 

...  :./v"  . / .signal 
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TABLE  9.  BASIC  EW  DEFENSIVE  MISSION  MEASURES  AND  SCORES 
(Continued) 


Mis  s ion 'Phase 

Measure 

Scores 

• ' ’ ’ ’ 

Mode  of  operation  in  switch 
settings 

receiver 

transmitter 

expendables 

Malfunction  correction 
sequences 

Total  number  of  signals 
countered 


Threat  assessment 


Switch  positions 
Time  of  handling  threat 


Setup  for  jamming 


Switch  positions  on  jamming 
systems 


Time  to  set  up 


Initiation  of  jamming 


Time  at  which  jamming  is 
initiated 


Time  during  which  jamming 
is  effective 


Use  of  expendables 


Switch  positions  for  selecting 
expendable  dispensing  programs 


Time  to  initiate 

Time  at  which  expendable 
program  is  terminated 

Failure  to  release 
expendables  •' 


: fp-r 


Maintenance  of  minimum 
stores  level  : : 


' 1®  "" 
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TABLE  9.  BASIC  EW  DEFENSIVE  MISSION  MEASURES  AND  SCORES 
(Continued) 


Mission  Phase  Measure  Scores 


Threat  priority  Signals  attended  to  in  time 

Switch  positions 

Time  of  handling  new,  higher 
priority  threat  (cue  buildup) 

Environment  coverage  Signals  and/ or  bands  covered 

Jammer  refinement  switch 
settings 

■ v . ■■  ,;k7;  Initial 

subsequent 

Number  of  jammers  on  indi- 
vidual  signals  and/or  bands 

Evasive  maneuvers  Maneuver  selection 

Time  of  initiation 

Communications  Switch  positions  on  interphone 

and  communications 
equipments 

Post-flight  shut-  Switch  shutdown  Sequence  of  switch  positions 

down  positions 

Switch  setting  error 

Communications  (position  of 
channel  switches) 


:;.V7 IDO 

O * V77H7\vV  .7  7:'.7.7-  : ' ■ :v;;777:, 

■^:EIUC';0:7V7  '7;7  ; rv"-7;V-  :;;-7;7.^ 
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TABLE  10.  PARAMETERS  RELEVANT  TO  MEASURING  FLIGHT  AND 
NAVIGATIONAL  SKILLS  IN  AN  OPERATIONAL  FLIGHT 
TRAINER 


Measure  Parameters  (Deviation  Recording) 


1.  Aircraft  attitude/ position  Heading,  altitude,  air  speed',  pitch 

geometry  angle,  roll  angle,  flight  path,  flight 

path  deviation,  approach  slope  , 
approach  slope  deviation. 

» ' . . - • - . 

2 . Force/ rate  G-  loading  5 vertical  velocity,  turn 

rate,  longitudinal  stick  rate,  lateral 
stick  rate,  rudder  rate,  pitch  rate, 
roll  rate,  yaw  rate. 

3.  Position  of  aircraft  controls  Longitudinal  stick  position,  lateral 

stick  position,  rudder  position, 
throttle  position. 

4.  Discrete  event  activation  Flaps,  landing  gear,  speed  brakes, 

thrust  attenuator , elevator  tab  up, 
elevator  tab  down,  canopy  controls. 

5.  Time  Time  to  initiate  an  event  or  sequence 

from  onset  of  signal,  time  to  com- 
plete an  initiated  activity,  overall 
time  from  signal  onset  to  activity 
completion. 

6.  Instrument  readings  Engine  ins truments  , RPM. 
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For  more  complex  performances,  the  criteria  should  be  based 
on  empirical  evidence.  Ideally,  experiments  and  actual  data  collection  in 
the  operational  setting  are  desired  for  establishing  the  standards.  Practi- 
cally, however,  expert  opinions  are  used  to  establish  criteria  for  success- 
ful performance.  This  intuitive  approaches  good  where  standards  do  not 
exist  or  have  not  been  quantified.  Interim  criteria  can  be  developed  in  this 
manner  since  subject  matter  experts  can  provide  solid  guidance  in  the 
development  of  standards.  The  weakness  of  this  approach  is  that  the  biases 
of  these  people  may  influence  the  standards  set. 

Where  a scoring  capability  exists,  obtaining  repeated  scores 
will  allow  the  development  of  criteria.  For  example,  if  an  actual  simula- 
tion of  the  system  is  set  up  in  the  course  of  its  development,  data  can  be 
obtained  to  identify  required  performance  levels.  . 

The  error  criteria  should  be  flexible  to  account  for  stage  of 
training,  and  modifiable  to  enable  change  as  experience  with  the  training 
program  is  gained.  Thus,  initially,  tolerances  are  set  intuitively  based 
on  previous  ;experience  and  judgment  and  are  refined  as  a data  base  accu- 
mulates with  experience  in  the  use  of  the  system  or  as  training  objectives 
are  modified. 

In  automated  training  sequences  (e.g.  , preprogramed  scenarios 
that  may  be  modified  by  the  instructor  during  an  exercise)  consideration 
must  be  given  to  the  capability  for  modifying  the  computer  scoring  criteria 
in  the  software  programs.  For  example , an  "error,  alert"  mode  is  a special 
case  of  evaluation  whereby  the  computer  halts  the  problem  when  a trainee 
exceeds  the  permissable  error  envelop  in  terms  of  the  preset  scoring  cri- 
teria established  for  a particular  mission  or  stage  of  training.  The 
requirement  is  to  set  in  new  error  criteria  to  establish  the  next  error  alert 
(computer  halt)  level.  This  can  be  accomplished  via  CRT  display  (e.g.  , 
Page(s)  of  alpha-numeric  information  for  error  analysis)  and  appropriate 
controls  (e.  g;  , keyboard,  light  pen).  A design  decision  concerns  the  pro- 
vision of  this  capability  at  the  instructor  console . The  instructor  may 
override  the  computer  halt  or  accept  the  halt  with  or  without  inserting  new 
evaluation  criteria,  in  order  to  implement  decisions  on  training  strategy 
(e.g.  , to  continue  the  exercise  as  briefed,  to  replay  a portion  of  the  just 
completed  training  segment  for  the  purpose  of  reinstruction,  or  to  provide 
demonstration  to  the  student)  . 

2 . 8 .9 . 4 Determine  Conditions  Under  Which  to  Measure  Performance  . 
Once  the  measurement  requirements  have  been  determined,  the  task  con- 
ditions under  which  performance  should  be  measured- must  be  specified. 

As  with  the  tasks  themselves,  the  conditions  under  which  performance  is 
observed  will  necessarily  represent  a sampling  of  the  anticipated  real 
conditions.  In  selecting  the  sample  of  conditions  that  will  prevail,  the 
following  two  general  rules  should  be  consi-^^fid: 
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Those  conditions  should-be  selected  which  are  known 
(or  suspected)  to  have  the  greater  influence  on 
performance.  » 

Within  those  conditions  selected,  the  extremes  or 
limits  anticipated  should  be  included. 

Task  conditions  are  independent  of  what  measures  will  be  used 
(i.e„  / decide  on  what  measurements  are  desired,  then  determine  the  con- 
ditions). The  conditions  of  measurement,  however,  do  influence  when  the 
measurements  will  be  taken  and  in  some  instances  how  often  the  measures 
will  be  repeated.  For  example,  when  load  conditions  vary  in  a mission 
sequence,  it  may  be  appropriate  to  take  measurements  at  selected  points 
in  order  to  indicate  system  status  or  performance  at  these  times. 

Emphasis  must  be  placed  on  the  standardization  of  test  condi- 
tions and  environments  . Not  only  should  measures  be  standard  (used  in 
similar  ways  with  the  same  meaning  by  the  personnel  involved)  but  also, 
the  general  environment  in  which  the  measurement  is  to  be  taken  should  be 
standard.  Adherence  to  this  insures  that  measures  will  be  strictly  com- 
parable from  one  student  to  another,  from  one  team  to  another,  and  from 
one  time  to  another.  > >' 

Two  types  of  variations  in  the  conditions  under  which  perfor- 
mance can  be  observed  are  apparent.  These  are,  task  conditions  and  en- 
vironmental conditions.  Measurement  should  be  conducted  under  a variety 
of  task  loadings  representative  of  those  which  may  be  or  are  encountered  in 
operations  . Task  loading  should  be  systematically  manipulated  to  induce 
stress  evoked  by  situations  requiring  unusual  or  complex  decisions  and 
emergency  situations  which  give  rise  to  additional  task  demands. 

It  is  important  also  to  conduct  measurement  under  the  more 
important  environmental  conditions  . These  include:  acceleration  forces  , 

movement  about  axes , sound  and  vibration,  temperature  extremes , and 
the  range  of  visual  conditions . The  selection  should  consider  those  that 
have  importance  to  successful  performance  and  will  not  endanger  the  safety 
of  the  individual.  This  sampling  should  include  the  extremes  expected  and 
also  unusual  conditions  . v . 

In  order  to  provide  the  selected  conditions  in  a uniform  manner 
throughout  traihing  and  measur em  design  should  provide  for  calibration 

of  the  simulated  environment./;  ; . ;■  ■,/. / .. 


2.8. 9 . 5 Data  Handling  and  Display.  Instrumentation  is  a key  factor  in 
the  design  of  a measurement  system  and  selecting  the  equipment  array  is 
a critical  aspect.  The  display  of  student  performance  information  and 
error  information  must  consider:  i/C/-.;.  / 
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a.  The  number  and  types  of  displays  required 


Counters,  timers  and  scalars  (meters,  CRTs) 
provide  classes  of  measures  describing  fre- 
quency, time,  accuracy,  amplitude  arid  amount. 


Computer  driven,  multi-format  Cathode  Ray 
Tube  displays  afford  the  greatest  versatility 
for  instructional  control. 


b.  The  display  modes 


Both  continuous  performance  error  indications 
as  well  as  discrete  classes  of  performance  . 
information  on-demand  are  required.  CRT 
displays  include  both  alpha-numeric  and 
graphic  page  formats  with  associated  selection 
controls  (e  . g.  . light  pen,  keyboard  input  devices)  , v; 


Computer  generated  error  alert  indication  pro- 
vides an  automatic  display  of  student  error 
information  when  performance  is  out-of- 
tolerance  for  a defined  error  class. 


2.8.9,.  6-  Techniques  for  Recording!  Measurement  Data.  Computer  pro- 
graming is  required  to  1)  monitor  the  critical  aspects  of  student  perfor- 
mance, 2)  score  all  students  objectively  against  standard  parameters , 

3)  relay  pertinent  performance  and  error  information  to  instructor  console 
CRT  displays,  4)  deliver  performance  cues  and  performance  information 
to  displays  at  each  student  station  (for  example,  augmented  feedback  of 
performance  information  in  the  form  of  alpha-numeric  error  indication), 
and.  5)  provide  a record  of  performance  for  all  students  throughout  each 
mission.  Since  the  digital  computer  is  capable  of  recording  all  time  and 
event  happenings  , various  records  may  be  provided  via  hard  copy  printouts 
The  permanent  records  should  be  considered  both  for  student  critique  and 
for  school  record  keeping.  Assuming  hard  copy  records  of  performance 
are  warranted,  the  design  issue  concerns  what  information  is  desired  and 
their  formats  . In  essence  , three  major  classes  of  records  are  indicated: 


Time  and  event  printouts  for  school  record  keeping 
and  for  developing  a normative  data  base  of  per- 
formance information.  (The  concern  for  quality 
control  in  the  production  of  trained  personnel  is 
a legitimate  issue  for  design.  Measurement  data 
provide  the  basis  for  determining  the  quality  level 
of  the  student  -population  trained  in  the  de vie  e . ) 
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. Error  printouts  for  mission  critique  (for  use  during 
and  after  exercise  completion)  in  terms  of  error 
classes  and  error  trends. 

. Summary  of  error  scoring- - single  page  summary 
of  errors  per  class,  scores  achieved,  and  total 
score  for  the  exercise.  A file  may  be  kept  of  all 
student  training  exercises  for  the  purpose  of  per- 
formance diagnosis . 

Measures  obtained  during  each  exercise  must  be  reduced  into 
a meaningful  usable  form  quickly  in  order  to  be  employed  for  critique  pur- 
poses. Thus,  the  data  reduction  system  must  be  an  integral  part  of  the 
device  computational  capability  and  must  be  flexible  to  allow  modifications 
in  measures  and  standards  as  experience  is  gained  with  the  measurement 
system . \ 

2.8. 9.7  Summary  of  Design  Requirements  for  a Measurement  System. 
Human  factors  design  must  consider  the  following  in  the  development  of  an 
automated  measurement  system . 

. Preprogramed  mission  scenarios  graduated  in 
exercise  difficulty; 

. Automated  monitoring,  evaluation  and  scoring  capability . 

. Definition  of  measures  and  standards. 

. Definition  of  data  collection  techniques  (see'/Figure  17); 

± Displays  of  student  performance  and  error  information 
in  all  relevant  modes  and  formats  at  the  instructor 
station.  ‘ ..  "■  ■■■■'■• 


Capability  for  recct 


ding  student  performance  information; 


. Capability  for  sampling  student  performances  at  specified 
points  or  intervals  of  time  during  an  exercise.: 

• Capability  for  computing  scores,  combining  of  scores 
arid  the  weighting  of  scores  according  to  the  estab- 
lished performance  criteria. 

. Hard  copy  printouts  of  student  performance  information 
for  exercise  critique  arid  fo>r  school' record  keeping. 
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CLASS  OF  CLASS  OF  CLASS  OF 

INFORMATION  MEASURE  . INFORMATION 

DISPLAY  . RECORDER 
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2.8.10  Communications.  The  communications  capability  at  the  instructor 
console  involves  the  following  considerations. 

• Extent  of  representation — replica  of  the  operational 
system  counterparts. 

• The  use  of  actual  system  communications  components 
(e.g. , controls  and  control  panels,  headsets,  micro- 
phones, etc.)  configured  and  operated  as  in  the  opera- 
tional system. 

• Instructor -student  communications --selective  communi- 
cation with  individual  students,  any  combination  of  students, 
all  students  on  interphone  (as  applicable). 

. Status  displays  depicting  (as  applicable)  which  students 
are  transmitting,  receiving  and  what  transmitters 
are  being  used;  call  lights. 

. Ability  to  introduce  static,  background  noises , back- 
ground and  simultaneous  communications  from  other 
sources . 

• Student -to -student  communications  (as  applicable). 

. Instructor  to  remote  instructor( s) / operator  communi- 
cations, unavailable  to  students  . 

_ Remote  instructor  to  student  links. 


. Instructor  monitoring  of  all  communications  channels. 

. Ability  to  introduce  extra-vehicle  communications 

(e.g.  , messages  from  other  units,  ground  control,  etc.). 

2.8.11  Overall  Layout  and  Workplace  Requirements.  Instructor  station 
layout  should  be  accomplished  in  modularized  components  consistent  with 
primary  and  secondary  use  requirements  and  with  established  human 
factors  engineering  design  practices.  Console  configurations  should  re- 
flect the  groupings  of  display  and  controls  to  accommodate  related  func- 
tions and  positioned  in  terms  of  frequency  of  use  and  criticality.  The 
design  must  also  consider  flexibility  in  usage  of  the  consoles,  i.e.  , accom- 
modating a single  instructor  or  simultaneous  use  by  more  than  one  ins  true - 
■ tor  . A general  idea  of  desired  layout  groupings  is  shown  in  Figure  18  . 

Special  design  problems  pertinent  to  a given  training  device  under 
consideration  must  also  be  resolved.  These  center  on  issues  arising  from 
constraints  in  layout  and  workplace,  due  primarily  to; 
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Figure  18.  General  Layout  of  Display  and  Control  Groups  in  Instructor 
Station  Design. 
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• Space  limitations  in  room  size  or  in  dimensions 
(such  as  aboard  a vehicle). 


» Arrangements  requiring  integration  of  components 
(e.g.  , incorporating  the  instructor  console(s)  into 
a master  briefing  room  which  may  involve  problems 
in  visibility  during  critique  sessions. 


Requirements  for  the  instructor  to  have  unlimited 
visual  access  to  all  students  as  they  perform. 


A representative  example  of  instructor  station  design  is  provided 
by  the  Synthetic  Flight  Training  System  (SFTS),  Device  2B24  (Hundt,  1969). 
Figure  19  is  an  artist’s  conception  of  the  SFTS  showing  the  instructor  station 
in  relation  to  the  four  trainee  cockpits.  Figure  20  shows  the  layout  of  the 
controls  and  displays  on  the  instructor  consoles.  The  layout  is  founded  on 
the  intended  primary  use  of  the  device  in. automatic  mode,  operated  typically 
by  one  instructor  with  numerous  visual  displays  available  to  several  train- 
ing personnel.  The  control  and  display  functions  on  each,  panel  are  arranged 
with  regard  to  expected  frequency  of  use  and  criticality. 

Another  example  of  instructor  station  layout  is  the  .design  proposed 
for  a generalized  submarine  advanced  casualty  ship  control  training  device 
(Lamb,  Bertsche  and  Carey,  1970).  This  complex  instructional  facility 
shown  in  Figure  21,  reflects  good  human  factors  design  practices  in  layout 
and  organization  for  achieving  instructional  eff: ciency . 

2.8.12  Output.  Functional  design  characteristics  are  specified  for  struc- 

turing, controlling  and  monitoring  training  at  the  instructor  station  complex. 
The  range  and  extent  of  instructional  functions  are  defined  in  terms  of  the 
display,  control  and  communication  requirements  to  achieve  the  capability 
for  the  management  of  training , commensurate  with  the  purpose  of  the  train- 
ing system.  = 

2.8.13  Human  Engineering.  It  is  sufficient  for  our  purpose  to  state  that 
the  basic  human  engineering  design  standards  and  military  specifications 
are  applicable  throughout  the  device  design  effort.  We  will  not  provide  a 
discussion  of  this  since  it  is  a substantial  design  area  possessing  a well- 
developed  body  of  literature  witb  ^well-established  principles  and  standards . 

A considerable  number  of  military  standards  and  specifications  are  in  use 
today  which  specify  the  human  engineering  requirements  for  training  device 
design.  Pertinent  documents  are  listed  below.  v. 


MIL-  D-  2 2 688  ( WE  P) 
1 November  I960 


Military  Specification  Data,  Avionics 
Design  Requirements  for  Developing 
Aviation  Training  Devices 
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Figure  19.  Artist's  Conception  of  Synthetic  Flight  Training  System,  Device  2B24. 
(from  Hundt,  1969) 
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1. 

Typewriter  :;.r 

16. 

Buoyancy  angle  indicator 

2. 

Performance  CRT  display 

17. 

Air  banks  pressure  Indicators 

3. 

Casualty  control  CRT  display 

18. 

OCT  tank  level  indicators 

4. 

T.V.  monitor 

19. 

NOR  MBT  (aft  and  fwd)  valve  position  Indicators 

5. 

Stern  planes  angle  indicator 

20. 

EM8T  blow  (aft  and  fwd)  vaive  position  indicators 

6. 

Planes  mode  indicators 

21. 

Vents  (fwd  and  aft)  position  Indicators 

7. 

Rudder~angle  Indicator 

22. 

Flooded  water  level  indicator 

8. 

Course  digital  read-out 

23. 

Propulsion  lost  indicator 

9. 

Pitch  angle  indicator  (left) 

24. 

Hydraulic  accumulators  contents  indicator 

io. 

Depth  indicator  (center) 

25. 

TAV  — average  steam  temperature  indicator 

11. 

Depth  rate  indicator  (right) 

26. 

Pump  on  indicators  — one  each  for  lead,  main,  and 

12. 

Speed  Indicator 

27. 

Light  pen  for  use  on  CRT  displays 

13. 

Fair  water  angle  indicator 

28. 

Trainer  controls 

14. 

Depth  to  bottom  indicator 

29. 

Ernergency  stop  control  button 

15. 

Horizontal  trim  angle  indicator 

Figure  21.  Instructor  Station  Design  Proposed  for  a Generalized 
Submarine  Advanced  Casualty  Ship  Control  Trainer  1 

rtcefesEj,  and  Carey,  1970). 

. . 1 ‘ :•  - - .ri!  i . * ■ 

i7s  ’ 

163  - . 


(from  Lamb,  Be 


O 

ERIC 


NAV TRADE VC EN  69 -C -0298-1 


MIL-D-2  3143(WEP) 
15  December  1961 

Military  Specification  Data,  Technical 
Aircraft;  For  Design  of  Aviation 
Training  Devices 

MIL-D-2  62  39A(USAF) 
14  April  1961 

Military  Specification  Data,  Qualitative 
and  Quantitative  Personnel  Require- 
ments Information  (QQPRI) 

MIL-D-27  382A(  USAF) 
31  January  1967 

Military  Specification  Data,  Training 
Equipment  Technical 

MIL-H-24148  (SHIRS) 
15  November  1965 

Military  Specification  Human 

Engineering  Requirements  for 

Bureau,  of  Ships  Systems  and  Equipment 

MIL- H-27894A(  USAF) 
9 January  1963 

Military  Specification  Human  Engineer- 
ing Requirements  for  Aerospace 
Systems  and  Equipment 

MIL-H-46855 
29  March  19  68 

Military  Specification  Human  Engineer- 
ing Requirements  for  Military  Systems, 
Equipment  and  Facilities 

MIL-H-8  1444(AS) 
1 September  1966 

Military  Specification  Human  Factors 
Engineering  Systems  Analysis  Data 

MIL- T- 6328 H 
9 September  1965 

Military  Specification  Trainers, 
Maintenance  , General  Specification  For 

MIL-  T-  9 2 1 3B(  USAF) 
28  June  1967 

Military  Specification  Trainer,  Flight 
Simulator,  Aircraft,  General  Requir  e - 
rne nts  For 

MIL- T-23.99.1C 
1 June  19  69 

Military  Specification  Training  Devices, 
Military:  General  Specification  For 

MIL- T-27474(  USAF) 
22  August  I960 

Military  Specification  Training  Equip- 
ment, Ground,  General  Requirements 
For 

MIL-T-82  335  (ONR/NTDC)  Military  Specification  Trainers,  Flight; 


2 6 July  I965 

General  Specification  For 

MIL-  T-82  335A 
11  April  1969  : 

Military  Specification  Trainer , Fixed- 
Wing,  Flight;  General  Specification  For 

*—t 

" L.tf  O ■ 
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MIL- T-82342B  Military  Specification.  Trainers,  Cock- 

2 January  1970  - pit  Procedures , Aircraft;  General 

Specification  For 

Military  Standard  Aircrew  Station 
Vision  Requirements  for  Military  ' 
Aircraft 

MIL-STD-830A-2(USAF)  ^Military  Standard  Human  Engineering 
1 December  1964  Design  Criteria  for.  Aerospace  Systems 

and  Equipment 

Military  Standard  Human  Engineering 
Design  Criteria  for  Military  Systems, 
Equipment  and  Facilities 


Air  Force  Systems 
Command  AFSC  Manual 
No.  80-3 
15  April  1967 


Handbook  of  Instructions  for  Aerospace 
Personnel  Subsystem  Design  AFSCM 
80-3 


MIL- STD-  147  2 
9 February  1968 


MIL- STD- 8 50 A 
8 June  1967 
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SECTION  III 


CONCEPTS  AND  DATA  APPLICABLE  TO 
THE  DESIGN  OF  TRAINING  SYSTEMS 

This  major  section  presents  concepts  and  data  which  support  the  human 
factors  effort  in- the  design  of  synthetic  training  systems.  It  is  based  on  a 
review  of  a substantial  number  of  studies-  available  in  the  literature  that 
deals  with  various  topics  pertinent  to  the  human  factors  design  require- 
ments articulated  in  Section  II  of  this  report.  These  researches  cover  a 
considerable  range  of  content  areas,  employ  diverse  methods  and  are  set 
in  various  laboratory,  simulation,  and  field  contexts.  Our  intent  is  straight- 
forward: it  is  to  examine  this  literature  and  determine  from  the  array  what 

data  and  concepts  are  serviceable  in  support  of  the  human  factors  design 
effort . 

Although  a considerable  body  of  information  and  data  has  accrued  over 
many  years  of  research  and  development,  the  unfortunate  feature  is  that  a 
data  base  has  not  evolved  which  provides  firm  guidelines  for  ■'She  human 
factors  design  of  training  devices.  Quantitative  design  data  are  meager, 
and  spotty;  some  design  areas  possess  little  in  the  way  of  supportive  data 
and  require  considerable  applied  research  to  acquire  the  information  needed 
for  design.  This  is  particularly  true  of  the  recent  innovative  approaches 
developed  for  training.  Much  of  the  available  data  that  is  pertinent  to  de- 
sign comes  from  laboratory  research  and  we  are  beset  by  the  traditional 
problems  of  correlating  laboratory  findings  with  the  more  complex  re- 
quirements involved  in  synthetic  training  for  real  job  situations  . For  the 
most  part,  laboratory  tasks  are  simplified  abstractions  of  the  real  job  and 
generalizing  from  this  is  done  with  significant  risks  . As  a result,  con- 
siderable reliance  in  training  device  design  has  been  placed  on  logical, 
intuitive  considerations,  and  on  empirical  evidence  . 

Accordingly,  our  approach  in  .this  section  is  to  assemble  from  the  liter- 
ature those  data  and  concepts  which  possess  utility  for  design  (albeit  dif- 
ferentially) . Also,  where  evidence  is  significantly  lacking,  the  data  gaps 
are  identified  so  that  the  discussions  can  be  placed  more  nearly  in  per- 
spective. Finally,  re  searchable  issues  that  are  of  priority  for  human  ; _ , 
factors  design  are  identified.  V 

Seven  chapters  are  presented  in  this  section.  They  deal  with  the 
■ following:  : ’ ■ ..  ji:/  .'-V 


O 

ERIC 


Visual  simulation 
Platform  motion  simulation 
Vehicle  control  requirements 
Information  processing  requirements 

Measurement  system  design 
Adaptive  training  strategies 

.Deliberate  departures  from  realism.in  design 


> 9 • Q 

I/O 

- 166  - 


NAVTRADEVCEN  69-C-0298-1 


3.1  VISUAL  SIMULATION. 


This  chapter  sets  out  the  considerations  affecting  the  design  of  the 
visual  environment  in  a training  device. 

The  visual  environment  usually  consists  of  some  enclosure  in  which  the 
trainee  sits  (for  example,  the  cockpit  of  an  aircraft)  and  a view  to  the  outside 
world.  In  recent  times  the  simulation  of  the  outside  world  has  become  more 
usual,  thereby  increasing  the  realism  and  the  training  potential  of  training 
devices . , 

It  does  not  follow,  however,  that  the  best  trainer  is  necessarily  the 
trainer  with  the  most  complete  rendition  of  every  aspect  of  the  real-life 
visual  environment.  A trainer  may  be  designed  to  teach  only  procedures; 
that  is,  the  trainer  would  be  a part-task  trainer.  A number  of  studies  have 
indicated  that  trainers  which  are  clearly  incomplete  in  their  simulation  of 
the  actuality,  can  be  effective  in  producing  an  increment  of  performance  in 
the  trainee  (Adams,  et  al,  I960;  Boney  and  Prophet,  1963;  Caro,  et  al, 

1968;  Dougherty,  et  al,  1957;  Ellis,  et  al,  1969;  Gagne  and  Foster , 1948) . 

Effective  visual  simulation  for  the  purposes  of  training  does  not 
depend  upon  achieving  a one-to-one  copy.  Rather,  the  aim  is  to  achieve 
an  adequate  representation  of  the  visual  environment  so  that  the  experience 
is  credible  to  the  trainee,  critical  visual  tasks  are  represented  accurately, 
and  all  changes  in  performance  (skill)  brought  about  in  the  trainer  have 
positive  transfer  to  the  actuality. 

In  fact,  it  may  be  asserted,  it  is  not  feasible  to  "recreate"  the  out- 
side world  in  a simulator  except  when  some  portion  of  reality  of  a rela- 
tively simple  kind  is  selected.  When  "recreation  of  reality"  is  regarded  as 
the  challenge  to  be  met  in  simulation,  an  inordinate  emphasis  is  placed  on 
elaborate  engineering.  The  true  challenge  is  to  meet  "task  fidelity"  re- 
quirements. Task  fidelity  may  be  defined  as  the  extent  to  which  the  training 
device  imposes  the  same  sensory,  motor,  and  data  processing  demands 
upon  the  trainee  as  does  the  actual  case.  The  consensus  of  much  human 
factors  evidence  is  that  when  there  is  an  equivalence  of  task  requirements 
there  is  effective  training.  : Is  ley  (1968)  reports  a critical  case  where 
physical  similarity  between  the  training  and  actual  situation  was  high,  but 
where  task  and  behavioral  demands  were  not  similar . Negative  training 
resulted.  f: 


^ This  chapter  on  Visual  Simulation  was  prepared  by  Dir.  Hugh  M.  Bowen 
of  Dunlap  and  Associates  , Inc  . , and  Dr . Knox  E . Miller  of  the  U.  S.  Naval 
Training  Device  Center.  Dr  .‘Miller  prepared  subsection  3.  1.3  which 
presents  a technique  for  developing  visual  simulation  specifications . 


Dr . Bowen  prepared  the  other  subsections  of 

■ rV 

x i d 


this  chapter  . 
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With  this  in  mind,  this  chapter  sets  out  to  relate  data  concerning 
visual  perception  to  the  various  operational  or  tactical  situations  most 
commonly  found.  In  this  way,  it  is  intended  that  the  designer  of  the  train- 
ing device  can  match  the  requirements  for  training  against  the  visual  char- 
acteristics appropriate  for  the  device. 

Data  concerning  vision  and  visual  processes,  related  as  specifically 
as  possible  to  simulator  conditions,  are  provided  first.  It  is  intended  that 
the  designer  should  be  able  to  turn  to  this  subsection,  starting  with  a visual 
topic  in  mind  (e.g.  , depth  perception),  and  find  useful  data  and  guidance 
concerning  that  visual  function  and  its  design  implications. 

Secondly,  operational  situations  are  analyzed  in  visual  simulation 
terms.  Typical  air-to-air,  air-to-ground,  on-ground,  and  in-water  situa- 
tions are  considered.  The  various  techniques  of  visual  simulation  are  re- 
viewed from  the  point  of  view  of  their  advantages  and  limitations  as  visual 
media.  In  addition,  a method  for  evaluating  simulation  techniques  for  a 
particular -.training  application  is  developed. 

Finally,  major  research  issues  in  the  field  of  visual  simulation  are 
discussed. 


3.1.2  Basic  Visual  and  Visual  Scene  Factors. 

3.  1.2. 1 Introduction.  The  visual  environment  is  defined  by  the  follow- 
ing major  visual  characteristics: 

• Size  of  field 

• Size- of  objects 

• Luminance  of  objects  .and  background 

• Shape  of  objects 

• Color  of  objects  and  background 

• Distance  of  objects  in  the  field 

• Motion  of  objects 

• Variations  in  brightnes  s 

These  characteristics  and  associated  topics  will  be  discussed  with  specific 
reference  to  simulation  considerations . 


3. 1.2.2  Size  of  Field.  From  the  observer's  point  of  view,  the  visual 
scene  is  limited  by  the  angles  of  sweep  generated  by  the  head  and  eyes  . 
Table  11  provides  the  pertinent  data . 


To  what  extent  is  it  necessary  or  desirable  to  provide  the  full 
extension  of  the  visual  scene  ? Data  indicate  the  importance  of  peripheral 
vision  in  maintenance  of  balance,  equilibrium  and  orientation  in  piloting 


(Berens  and  Shephard,  1952;  Hopkin,  1959) 


Adequate  scene  perception 
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TABLE  11.  THE  LIMITS  OF  THE  VISUAL  FIELD  UNDER  VARIOUS  KINDS  OF  RESTRAINT 
(Continued) 


NAV  TRADE  VC  EN  69-0 -0298-1 


NAVTRADEVCEN  69-C-0298-1 


is  necessary  to  overcome  possible  false  impressions  from  body  tilt  (Witkin 
and  Asch,  1948(a),  1948(b)).  The  adequate  perception  of  motion  is  depen- 
dent upon  the  perception  of  a broad  scene  (Gibson,  1955)  and,  particularly, 
in  such  tasks  as  landing  an  aircraft  (Calvert,  1954,  1955).  Salvatore's 
(1968)  studies  suggest  that  motion  may  be  judged  more  accurately  in  the 
periphery  than  foveally.  Luria's  (1968)  data  demonstrate  that  peripheral 
vision  contributes  to  stereo-acuity.  There  is,  therefore,  convincing 
evidence  pointing  to  the  utilization  in  visual  perception  of  the  visual  peri- 
pheral inputs . 

Typically,  the  workplace  enclosures,  such  as  cockpit  structure , 
tend  to  limit  the  field  of  view  to  the  outside.  It  is  generally  considered  de- 
sirable to  "fill"  the  window  scenes  with  scenic  material  whenever  possible. 
Conant,  et  al,  (1962)  caution  that  a cutoff  in  the  periphery  gives  not  only  a 
false  appearance,  but  a false  cue.  It  is  true  that  an  aircraft  can  be  flown 
and  landed  with  as  little  as  10°  forward  vision  provided  by  a periscope 
(Roscoe  and  Williams,  1949;  Roscoe,  1951).  It  has  been  argued  that  this 
result  justifies  a limitation  of  scene  in  a simulator.  This  would  seem  to 
argue  that  because  we  may  be  able  to  manage  to  get  around  with  blinkers 
on,  there  is  therefore  no  difference  between  normal  vision  and  narrowed 
vision. 

However,  the  fact  that  peripheral  vision  contributes  significantly 
to  visual  perception,  and  that  peripheral  scenes  are  necessary  to  provide 
credibility  in  the  scene  , should  not  be  allowed  to  detract  from  the  obvious 
fact  that  the  major  portion  of  the  visual  input  is  gained  in  central  (foveal) 
vision.  Landing  an  aircraft,  particularly  on  a carrier  in  the  open  sea,  is 
a case  where  task  information  is  concentrated  in  central  and  parafoveal 
vision  (Wulfeck,  et  al,  1958;  Calvert,  1955).  For  such  cases  where  task 
fidelity  may  be  preserved  with  a small  visual  field,  a simulation  of  only 
the  visual  target  area  which  appears  in  the  central  6°  visual  cone  may  be 
acceptable  on  the  basis  that  training  effectiveness  can  be  achieved  by  this 
means . 

A further  consideration  of  "size  of  field"  is  the  size  of  what  is 
termed  the  "Visual  World  Envelope."  The  question  is- -how  big  a piece  of 
‘ the  visual  world  is  it  necessary  to  present  to  the  trainee?  The  answer 
depends,  in  the  case  of  pilotage,  on  the  mission,  speed,  altitude,  time 
period  of  flight,  horizontal  flight  path,  time  of  day/ night,  etc  . Typical 
data  reported  by  Aronson  ( 19 63)  are:  - 

„ Helicopter  navigation.:/  airspeed,  0- 140  knots;  altitude, 

200  to  600  feet  above  terrain;  time  of  flight,  up  to  one 
hour;  lateral  fly  area,  1-1/2  miles  either  side;  maximum 
/ visible  distance,  up  to  5 miles. 
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• Fixed  wing  aircraft:  low  visibility,  landings  and  takeoffs; 

airspeed,  0-250  knots;  altitude,  0-1000  feet  above  terrain; 
total  distance  in  direction  of  flight,  5-1/2  miles;  lateral 
fly  area,  1/2  mile  either  side. 

• Low  Altitude  High  Speed  Navigation:  airspeed,  600  knots 

and  less;  altitude,  50  to  200  feet;  total  distance  in  direc- 
tion of  flight,  120  miles;  maximum  visible  range,  up  to 

5 miles;  lateral  fly  area,  1 mile  either  side. 

These  typical  specifications  derive  from  an  analysis  of  the 
system's  operation  in  the  particular  circumstance  which  is  to  be  trained. 

It  is  necessary  to  specify  these  particulars  in  every  case  to  arrive  at  the 
desirable  "visual  world  envelope"  in  any  particular  case. 

3.  1.2.3  Size  of  Objects.  The  requisite  size  of  objects  in  visual  simu- 
lator scenes  is  a troublesome  problem  because  the  discrimination  capacity 
of  the  eye  (visual  acuity)  is,  under  the  best  circumstances , extraordinarily 
fine.  Acuity  resolutions  down  to  a few  seconds  of  arc  are  reported  (Graham 
1965).  However,  acuity  under  normal  viewing  conditions  is  seldom  better 
than  a minute  of  arc  and  is  often  appreciably  less.  The  data  of  Blackwell, 
et  al  (1959),  are  often  reported  in  this  context.  Bliss  (1966),  writes  of 
it:  "This  is  the  classic  report  on  this  subject.  No  one  fails  to  cite  it,  and 

no  one  knows  what  to  do  with  the  data,  since  it  is  the  best  illustration  extant 
of  a hopeless  discrepancy  between  results  of  carefully  controlled  simula- 
tion tests  and  carefully  controlled  flight  tests  against  the  same  targets." 

The  discrepancies  may  be  illustrated  by  the  overall  averages 
for  all  runs  in  the  Blackwell,  et  al  (1959)  studies: 


Average  Slant  Range 

Average  Probability 

Item 

at  Recognition 

of  Recognition 

Simulation 

15,895  feet 

0.89 

Flight  test 

11, 008  feet 

0,60 

It  has  been  rather  widely  assumed  that  targets  need  to  subtend 
12-20  minutes  of  arc  to  be  recognized.  For  instance,  Steedmcn.  and  Baker 
(I960)  found  that  search  time  for  objects  became  constant  when  their  size 
was  over  12  minutes  of  arc . A recent  and  thorough  field  study  (Gilmour, 
1964;  Gilmour  and  Tula  no,  1964)  wherein  observers  searched  for  a radar 
van  in  an  open  area  of  desert  country  at  a height  of  200  feet  above  terrain, 
at  noon,  at  a speed  of  mach  0.8,  indicated  that  effective  recognition 
occurred  when  the  subtense  angle  was  approximately  1°.  At  this  level,  one 
is  dealing  with  human  perceptual  and  interpretational  capability  rather  than 
simple  visual  acuity.  However  , as  far  as  the  designer  of  the  visual  scene 
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is  concerned. 

when  an  object  must  be  made  visible,  it  is  immaterial  pre- 

cisely  what  visual/ perceptual  mechanisms  are  involved. 

Certain  general  guidance  can  be  offered  to  set  limits. 


• 

Visual  acuity  increases  only  slightly  when  brightness  is 
above  10  footlamberts.  Below  this  level,  visual  acuity 
lessens  at  an  accelerating  rate  as  brightness  lessens 
(Moon  and  Spencer,  1944)  . 

• 

Dynamic  visual  acuity  (DVA)  , that  is  , acuity  for  moving 
targets,  is  less  than  that  for  stationary  targets.  DVA 
lessens  as  a function  of  speed  of  motion.  A typical  datum 
point  is:  when  visual  acuity  is  2 minutes  of  arc  for  a sta- 

tionary target,  the  target  size  has  to  be  increased  to  10 
minutes  of  arc  when  it  is  moving  at  140°  per  second 
across  the  observer's  line  of  sight  (Ludvigh  and  Miller, 
1958). 

• 

m 

Textured  ground  scenes  (or  symbolized  scenes,  such  as 
random  light  and  dark  squares)  and  patterned  cloud  layers 
possessing  various  levels  of  fineness  of  detail  provide  ade- 
quate visual  orientation  cues  (General  Electric  Company, 
1961). 

As  brightness  contrast  improves  , visual  acuity  also  im- 
proves, almost  proportionately  (Cobb  and  Moss,  1954). 
Luxenberg  and  Bonnes  (1965)  offer  as  ‘'acceptable  contrast 
ratios"  on  displays: 

Contrast  Ratio 

White  symbols  on  black  ground  5:1 

Line  drawings  or  text  on  a white 
background  25:1 

Pictorial  scenes  - 100:1 

(It  is  questionable  whether  a contrast  of  100:  1 is  feasible 
on  many  types-  of  displays  (Poole,  1966)  . ) 

Target  recognition  on  TV  or  other  displays  (e . g . / electro- 
luminescent displays)  is  affected  by  the  number'  of  shades  of 

ERIC 

gray  and  the  number  of  lines  defining  the  object.  Visual  per 
formance  becomes  approximately  constant  when  the  number 
; of Jtii s ce r nib le  shades  of  gray  is  8 (Johnston,  1968)  and  when 
an’^object  (symbol)  is  defined  by  at  least  8 lines  (Shurtleff, 

185  *••••'  . . - ■ 
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3.  1.2.4  Luminance  of  Objects  and  Background.  The  eye  adapts  to  the 
brightness  of  the  scene.  The  important  visual  functions  (acuity,  color 
perception,  depth  perception,  etc.)  reach  an  approximate  constancy  when 
the  liminance  is  1 to  10  footlamberts  (Berry,  et  al,  1950;  Lythgoe,  1952). 

To  simulate  daylight  conditions  , exact  physical  simulation  is  not  required 
to  give  the  impression  of  realism  (IPuig,  1970).  Both  brightness  contrast 
and  color  contrast,  serve  to  perceptually  articulate  the  scene  so  that  ob- 
jects, textures,  patterns,  etc.,  may  be  seen.  Thus,  perceptual  fidelity 
may  be  achieved  at  low  luminance  levels  if  the  proper  contrast  ratios  are 
provided.  Buddenhagen  and  Wolpin  (1961)  point  o^^t  that  when  such  effects 
as  dark  or  brightness  adaptation,  flash  blindness,  and  glare  discomxoi't 
are  being  simulated,  brightness  scaling  problems  occur.  Interpolation 
along  the  visual  functions  as  described  in  standard  texts  (e.g.  , Graham, 
1951)  is  advised.  Table  12  provides  a listing  of  representative  bright- 
nesses of  natural  objects  covering  the  entire  range  of  brightnesses  occur- 
ring naturally.  When  display  of  ground  scenes  is  desired  and  realism  is 
being  sought  it  should  be  remembered  that  the  apparent  brightness  (lumin- 
ance) of  objects  depends  upon  the  intensity  and.  direction  of  the  illumination, 
the  reflectance  of  the  objects,  and  the  viewer's  position  with  respect  to  the 
object. 

The  visual  and  optical  characteristics  of  objects  and  substances 
have  been  measured  or  estimated  as  follows  (Gordon,  1964): 

Typical  midday  outdoor  brightnesses  are: 

9000  footlamberts 
4500  footlamberts 
1800  footlamberts 
7 00  footlamberts 
67  5 footlamberts 

The  range  of  luminances  in  ordinary  outdoor  scenes  averages 
about  160:1,  although  extremes  of  760:1  have  been  observed  (Jones  and 
Condit,  1941).  In  specific  cases,  such  as  specularly  reflectant  objects, 
much  higher  extremes  occur. 

Data  concerning  color  of  terrain  has  been  reported  using  the 
Munsell  notation,  and  is  shown  in  Table  13. 

To  these  aspects  of  the  world  which  are  natural,  one  must  add 
cultural  features.  In  general,  it  is  recommended  that  the  density  of  cul- 
tural features  provided  be  the  same  as  occurs  in  the  type  of  landscape  being 
simulated.  A desert  or  the  arctic  has  few  cultural  features,  although  the 
few  that  are  present  will  tend  to  be  conspicuous.  An  urban  area  is  domi- 
nated by  cultural  features.  The  grain  of  1 minute  arc  is  considered  gener- 
ally acceptable,  although  a finer  grain  may  often  be  desirable. 

O 

ERIC 


- * 

ido 


Bright  Sunlit  Cloud 
Sandy  Beach,  Dry 
Barren  Soil,  Light 
Green  Grass  in  Sun 
Dark  Soil,  Dry 


174 


NAVTRADEVCEN  69-C -0298-1 


TABLE  12.  REPRESENTATIVE  BRIGHTNESS  OF 
NATURAL  OBJECTS 


Luminance 
in  ml 

Objects 

Notes 

1 X 109 

Sun 

Sun  viewed  from  outs  .de  earth's  atmosphere; 
7 x 108  ml, 

Sun 

Sun  viewed  from  earth:  4.4  x 108  ml. 

1 x 108 

A-bomb 

Fireball  4 miles  from  800  KT  weapon: 
8 x 107  ml. 

1 x 107 

106 

* 

105 

Upper  limit  of  visual 'tolerance:  1. 6 x 104  ml. 

| . . - ' 

VeiTUS 

Albedo  approx.  0.6;  viewed  from  outside 
atmosphere:  1.6  x 104  ml. 

104 

Earth 

With  cloud  cover;  from  space;  albedo  0.8; 
9.4  x 103  ml. 

Earth 

No  clouds;  from  space;  albedo  0.4;  4.3  x 103 
ml . 

103 

Sky 

Average  sky  on  clear  day:  2 x 10 3 ml. 

Moon 

Full  moon  viewed  from  earth:  8 x 10 2 ml. 

102 

Sky 

Average  sky  on  cloudy  day:  5 x 102  ml. 

Earth 

"Average"  earth  on  cloudy  day:  approx. 

1 x 102  ml. 

101 

White  paper 

In  good  reading  light:  2 x 10 1 ml. 

Mini  screen 

Approx.  16  ml.  TV  screen:  approx.  10  ml. 

1 

White  paper 

1 foot  from,  ordinary  candle:  1 ml. 

Snow 

In  light  of  full  moon:  8 x 1 0 “ ^ m.1 . 

O 

ERLC 


id  i 


vox 
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TABLE  12.  REPRESENTATIVE  BRIGHTNESS  OF 
NATURAL  OBJECTS  (Continued) 


Luminance 
in  ml 

Objects 

Notes 

10"1 

Limit 

readable 

Brightness  level  for  cha  Is,  etc.:  approx. 

8 x 10"2  ml . 

Lower  limit 

For  useful  color  vision:  2 x 10~^  ml. 

10-2 

Earth 

From  space  in  moonlight:  7.5  x 1 0 ~ ^ ml. 

10"3 

Night  vision 

Upper  limit:  1 x 10 ml . 

10"4 

Snow 

In  starlight:  9 x 10"^  ml. 

Green  grass 

In  starlight  and  earth  seen  from  space  in 
starlight , air  glow,  and  zodiacal  light: 

3 x 10~5  ml. 

10-5 

Lower  limit 

For  night  vision;  moonless  night  sky: 
1 x 10"5  ml. 

10"  6 

Space 

1 x 10"6  ml . 

/ • 
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TABLE  13.  TERRAIN  AND  SOME  VISUAL  CHARACTERISTICS 
OF  A TYPICAL  DESERT1 


Terrain  Type 

Visual  Characteristics 

Serrated  Mountains 

Reddish  brown 
2.5YR4/2,  4/3,  5/2,  5/32 

Rounded  Mountains 

Dark  gray  to  dark  brown 
2.5YR  2/2 , 3/2,  4/1 

Badlands 

Dirty  gray 
10YR  6/1,  5/2 

Wash 

Light  tan  to  medium  gray 
10YR  6/  1 , 6/2  , 6/3 

Desert  Pavement 

Dark  brown  or  black 
10YR  3/2,  2/2 

Gravel  Flats 

Light  tan  to  light  brown 
10YR  7/4,  6/4 

Sand  Plains 

Light  tan 
10YR  7/3,  7/4 

Bottom  Lands 

Gray-brown  or  buff 

10YR  5/3,  6/2,  6/3  when  dry 

10YR3/2,  3/3  when  wet 

1Y uma , Arizona  desert. 


^Munsell  notation;  hue,  value  and  chroma. 
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3.1.2. 5 Shape  of  Objects  . Shapes,  forms,  patterns  , etc  . , vary  in  how 
easily  they  are  perceived  as  a function  of  a multiplicity  of  quantitative  and 
qualitative  factors.  The  conspicuity  of  a shape  depends  on  the  perceptual 
set  of  an  observer.  For  example,  in  an  applied  setting,  Sturm,  et  al  (19  66), 
found  that  in  visual  contact  navigational  flight,  the  predesignation  of  targets 
and,  checkpoints  served  to  increase  navigational  performance  by  20  percent 
(target  acquisition  and  recognition)  . Predesignation  was  achieved  on  radar 
or  on  navigational  displays  in  the  cockpit. 

The  visibility  of  an  object  depends  in  complex  ways  on  its  scenic 
context.  Camouflage  purposefully  exploits  the  possibilities  of  merging 
"figure"  into  "ground,"  and  targets  may  be  ••naturally"  lost  when  imbedded 
among  irrelevant  forms.  A useful  review  and  survey  of  much  qualitative 
description  of  these  effects  is  provided  in  Wulfeck  and  Taylor  (1957)^. 

These  cautions  concerning  "real-life"  factors  in  the  perception 
of  form  are  necessary  in  order  to  qualify  the  geometric  and  other  factors 
of  the  form  itself  which  determine  its  conspicuity.  A form  is  seen,  not 
only  as  a form,  but  as  an  object  distinguished  from  its  background.  Thus  P 
the  greater  the  brightness  and/or  color  contrast,  the  more  conspicuous  the 
object  (Wulfeck  and  Taylor,  1957). 

Objects  tend  to  be  more  conspicuous  when: 

m the  object  is  large  (Moler,  1962;  Taylor,  1954; 

Miller,  19  64), 

. the  object  is  singulated  in  the  scene  (Boynton,  1957; 

Deese,  1956;  Gilm'our  and  Iulano,  1964), 

, the  object  is  sharply  defined  (absence  of  blur)  (Enoch, 

1958;  Fiorentini,  1961;  Fry  and  Enoch,  1959)? 

. the  object  forms  a simple  figure,  such  as  square, 
circle,  triangle,  straight  line,  etc.  (Collier,  1.931; 

Helson  and  Fehrer,  1932;  Kleitman  and  Blier,  1928; 

Whitner,  1933;  Casperson,  1950;  Bowen,  et  al,  I960), 

m the  object  conforms  to  some  or  all  of  the  following 
prescriptions: 

- simplicity 
symmetry 
continuous  contour 
relatively  large  enclosed  area 

either  sharply  angular  or  simple  curve  forms 
familiarity--especially  in  the  sense  of  having  a 
/ common  name  or  meaning  (Bowen,  et  aL,  I9  60). 
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The  comprehensive  study  by  Gilmour  and  Iulano  (1964)  provides 
representative  data  on  the  relative  conspicuity  of  several  targets  (Table  14). 
The  tests  were  conducted  in  a jet  aircraft  simulator  with  the  observers 
viewing  a 35  mm  color  motion  picture.  Realism  was  gained  by  having  the 
movie  film  record  a flight  over  a 160-mile  course  at  three  heights  (500, 

400  and  200  feet)  and  at  two  speeds  , mach  0 . 85  and  mach  1.2. 


TABLE  14.  CONSPICUITY  OF  TARGETS 


Target 

Average 
Acquisition 
Range  ( feet) 

Probability 

of 

Acquisition 

Row  of  mothballed  freighters  in  a bay 

12,160 

1.00 

Long,  low,  causeway  type  bridge. 

8,000 

0.86 

Large  warehouse  on  dock 

7,800 

0.94 

Bridge  and  causeway  across  larg#  river 

7,660 

0.97 

Isolated  radio  tower  and  shack 

7,140 

0.94 

Docked  battleship 

5,590 

0.81 

Plywood  processing  plant 

4,  540 

0.89 

Gasoline  storage  tanks 

3,590 

0 . 89 

Small  factory 

3,510 

0.72 

Radio  communications  station:  buildings 

and  antenna 

3,400 

0.72 

(from  Gilmour  and  Iulano,  1964) 


3.  1.2.6  Color  of  Objects  and  Background.  Color  is  a fact  of  visual  real- 
ity and,  hence,  realism  is  enhanced  when  a visual  scene  appears  in  color  . 
Color  may  be  an  absolute  requirement  in  achievement  of  task  fidelity  in 
training  situations,  such  as  search  for  color-coded  life-rafts,  perception 
of  color-coded  pyrotechnics,  the  simulation  of  nighttime  lighting  for  take- 
off and  landing  (green  boundary  lights,  red  obstruction  lights,  white  run- 
way lights,  blue  taxi  way  lights,  etc.). 


Color  contrast  (with  equal  brightnesses  of  object  and  background) 
can  be  equivalent  to  a brightness  contrast  of  35  percent  (Eastman  Kodak, 
1958).  Hence,  the  use  of  color  generally  increases  the  amount  of  informa- 
tion transmitted  by  a display  (Hitt,  1961;  Smith,  1963,  Smith  and  Thomas, 

19  64).  Color  contrast  is  mainly  dependent  on  the  saturation  of  the  color. 

The  more  saturated,  "pure,  11  the  color,  the  more  contrast  it  has  to  its 
surround.  Color  contrast  is  affected  by  brightness  (too  low  or  too  high  a 
brightness  are  both  unfavorable  to  effective  color  contrast  (Purdy,  1931)) 
and  by  the  size  of  the  colored  patch.  For  underwater  simulation  it  should 
be  noted  that  ocean  water  acts  as  a blue/ green  filter  gradually  eliminating 
^~d  as  depth  increases  . An  exception  can  occur  in  very  murky  river 
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waters  where  all  colors  may  be  shifted  toward  red  (Kinney  et  al,  1967). 
Conover  and  Kraft  (1958)  suggest  that  color  recognition  should  not  be 
counted  on  for  objects  subtending  less  than  20  minutes  of  arc. 

3.  1.2.7  Distance  of  Objects  in  the  Field.  Actual  perception  of  volu- 
metric space  is  achieved  by  a combination  of  binocular  and  monocular  cues. 
In  nearly  all  simulations,  the  volumetric  space  is  much  constricted  and 
visual  space  has  to  b e recreated.  Binocular  cues  can  be  recreated  only 
by  separating  the  images  at  the  two  eyes  (see  Vlahos  (1965),  for  a survey 
of  techniques  and  methods).  While  several  methods  are  available,  they 
have  been  seldom,  if  ever,  used  in  training  devices.  The  probable  reasons 
are  ( 1)  the  incompatibility  of  the  stereoptic  condition  with  normal  viewing 
(e.g.  , colored  or  polaroid  glasses,  curved  lens  screens,  etc.)  and  (2)  that 
very  powerful  depth  impressions  can  be  obtained  from  monocular  cues. 


The  first  difficulty  in  simulation  of  visual  depth  is  that  the  scene 
is,  in  fact,  at  some  physical  place  close  to  the  observer.  Virtual  image 
techniques  remove  the  image  to  optical  infinity,  but  these  techniques  are 
still  confined  to  relatively  small  scene  sizes.  When  physical  projection 
screens  are  used,  a common  practice  has  been  to  place  the  screen  10  feet 
or  so  from  the  observer  (Aronson,  1965).  However,  eye  convergence  and 
eye  accommodation  approach  optical  infinity  conditions  for  objects  at  18 
to  20  feet  (Wulfeck,  et  al,  19  58).  However,  screens  placed  20  feet  from  the 
observer  increases  greatly  the  requirements  for  brightness  of  the  display 
generating  sources,  equipment  size  and  space.  Experience  suggests  that 
a distance  between  10  and  20  feet  is  satisfactory  and  is  a sensible  compro- 
mise with  other  requirements.  The  perception  of  the  actual  visual  location 
of  the  scene  is  minimized  with  large-angle  screens  and  absence  of  any 
visual  structure  between  the  observer  and  the  scene  (Puig,  1970). 


Monocular  cues  to  depth  include: 


relative  size, 
relative  brightness, 

relative  definition  (increasing  blur  with  distance), 
texture  gradient, 
linear  perspective, 
interposition, 

shadows  and  shading  effects, 

motion  parallax- -when  observer  is  moving,  near 
objects  appeaTr  to  move  across  distant  objects, 
expansion  and  streaming  when  observer  is  in  motion. 


Any  one  or  a combination  of  these  cues  can  be  the  dominant  cue,  depending 
on  circumstances.  It  Is  unsatisfactory  to  have  cue  conflict  or  lack  of 
realism.  A common  source  of  lack  of  realism  derives  from  use  of  models 
and  the  absence  of  aerial  perspective  (haze  and  blur)  . Particular  attention 
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needs  to  be  paid  to  this  feature  owing  to  the  often  dominating  role  that 
aerial  perspective  plays  in  depth  judgments  (General  Electric  Company, 

1961). 

A special  case  is  the  simulation  of  underwater  conditions. 
Visibility  is  restricted  because  of  transmission  attenuation  of  light  energy 
through  water.  Typical  data  are  presented  in  Table  15. 


TABLE  15.  TRANSMISSION  ATTENUATION  OF  LIGHT 
ENERGY  THROUGH  WATER 


Attenuation  Coefficient  ( tt) 

Distance  at  Which 
20%  of  Light  Energy  Remains 

0.05  (exceptionally  clear  water) 

35  meters 

0. 10  (common  tap  water) 

15  meters 

0,33  (typical  bay  and  estuarine  waters) 

6 meters 

(after  Loucks,  1970) 


3.  1.2.8  Motion  of  Objects.  An  object  moving  at  about  2 minutes  of  arc 
per  second  against  a stationary  background  will  just  be  seen  as  moving 
(Graham,  1965),  provided  the  target  is  seen  for  a long  enough  time--gener- 
ally  1 or  2 seconds  - -and  the  scene  is  bright  enough--10  footlamberts  or 
more  (Liebowitz,  1955).  In  general,  the  brighter  the  scene  and  the  longer 
the  viewing  period,  the  easier  it  is  to  perceive  motion.  Motion  is  also 
readily  seen  in  the  periphery  but,  due  to  the  decrease  of  acuity  in  the  peri- 
phery, somewhat  higher  rates  of  motion  are  required  to  achieve  threshold 
values.  As  compared  to  2°  per  second  foveally,  the.  threshold  for  motion 
at  70°  to  80°  in  the  periphery  is  approximately  S°  per  second  (McColgin, 
1960).  * 

Rates  of  change  of  speed  are  also  important  to  judge  in  many 
control  tasks,  such  as  landing  an  aircraft.  When  the  object  is  moving 
slowly,  1°  to  10°  per  second,  an  instantaneous  rate  of  change  of  about 
0. 15°  per  second  can  be  perceived;  at  higher  object  rates,  up  to  25^  per 
second,  instantaneous  changes  of  0.2°  to  0.25°  per  second  are  threshold 
values  (Hick,  1950;  Notterman  and  Page,  1957).  / 

Objects  moving  fast  enough  may  blur.  Graham  (1951)  reports 
that  under  the  best  of  conditions  an  angular  velocity  of  30°  per  second  with 
respect  to  the  viewing  axis  will  cause  blur.  Under  less  favorable  conditions 
15°  per  second  will  cause  blur.  Miller  (1964)  has  examined  the  blur  areas 
experienced  by  a pilot  in  a low-flying  aircraft.  However  , the  "blur  maps’1 
which  are  developed,  assuming  that  the  pilot  is  lookihg  straight  ahead,  are 
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somewhat  unrealistic  because  an  object  of  interest  will  be  fixated  and 
visually  tracked. 


Apparent  motion  may  be  induced  by  a variety  of  effects.  In 
general,  whenthe  physical  situation  replicates  the  conditions  of  a real 
motion  to  a sufficient  degree,  the  observer  mistakenly  sees  motion.  A 
walk  in  an  urban  district  at  night  will  demonstrate  the  varieties  of  effects 
that  can  be  produced  by  neon  signs.  Graham  (3  955)  discusses  the  conditions 
necessary  to  produce  apparent  motion  effects. 

Motion  of  the  field,  produced  by  observer  motion  through  the 
field,  produces  the  kinetic  cues  of  flow,  motion  parallax,  expansion- 
patterns,  etc.  (Gibson,  1955).  In  addition,  the  motion  has  effects  on 
visual  performance.  Search  performance  in  the  visual  periphery  was 
found  by  Erickson  (1964)  and  Williams  and  Borow  (1963)  to  be  impaired  at 
field  movements  greater  than  7°  to  8°  per  second.  However , various  op- 
erational studies,  for  example,  Joska  (1953),  Gilmour  (1964)  and  Gilmour 
and  Iuiano  (1964),  have  suggested  that  speed  over  the  ground  has  little 
effect  ori  search  performance,  within  the  range  of  speeds  studied  (mach  0.4, 
0.6,  0.8,  1.0, (1.2  in  the  Gilmour  study).  It  is  thought  that  observers  may 
compensate  for  blurring  and  time -stress  effects  by  modifying  their  visual 
search  patterns  . 


A common  practical  simulation  problem  is  the  presentation  of 
the  changing  visual  scene  when  landing.  Calvert  (1954 , 1955)  stresses  the 
importance  of  the  "form"  cues  and  the  transformation  which  the  runway 
geometry  goes  through  as  a function  of  the  travel  of  the  aircraft.  Gibson 
(1950)  emphasizes  the  Mfocus  of  expansion"  and  the  streaming  effects  as  the 
aircraft  is  flown  down  the  glide  slope  to  touchdown.  Payne,  et  al  (1954) 
report  that  pilots  feel  that  ground  texture,  perspective  cues,  and  opposed 
motion  of  ground  objects  are  the  significant  visual  cues  in  landing.  The 
consensus  would  appear  to  be  that  pilots  may  and  sometimes  do  use  all  of 
the  many  cues  in  the  scene.  The  pilot  sees  the  scene  as  a whole  and  forms 
his  ongoing  percept  from  combining  the  information  available  to  him. 

While  he  can  perform  with  diminished  cues,  Roscoe  (1951)  has  shown  that 
his  performance  suffers. 
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3.  1.2.9  Variations  in  Brightness  (Temporal  Factors).  The  perception 
of  motion  may  be  considered  to  be  a special  case  of  variation  of  the  scene 
or  scene  elements  over  time.  Other  effects,  often  of  an  unwanted  nature 
but  sometimes  useful  for  display  encoding  purposes  , are  flicker,  glitter, 
strobing  and  fading. 

Flicker  is  a major  source  of  brightness  interference  for  simu- 
lation designers  . Changes  in  brightnes s , caused  by  display  repetition  rates, 
scanning  techniques,  and  pattern  spacing,  may  cause  flicker  . In  addition 
to  impairing  image  fidelity,  flicker  causes  impairment  in  perception  and 
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may  occasion  nausea,  dizziness  and  fatigue.  Critical  Fusion  Frequency 
(CFF)  is  the  rate  above  which  an  intermittent  source  is  not  seen  to  be 
intermittent.  CFF  is  determined  by  a multiplicity  of  factors,  including 
the  following  (Ross,  1936;  Hecht  and  Schlaer,  1936;  Graham,  1965): 

• the  individual  observer 

• the  age  of  the  observer 

. the  location  on  the  retina  stimulated 

. the  area  of  retina  stimulated 

• the  duty  cycle  (on/off  ratio)  of  the  source 

. the  color  of  the  source  (at  low  light  levels) 

. brightness. 

Of  these  determinants  of  CFF,  the  single  most  important  factor  is 
brightness  . 

At  1 footlambert,  CFF  is  approximately  32  c.p.s 

At  10  footlamberts,  CFF  is  approximately  43  c.p.s 

At  100  footlamberts,  CFF  is  approximately  55  c.p.s. 

Various  special  forms  of  flicker  may  occur.  A relatively 
common  one  is  the  stroboscopic  interaction  of  the  moving  eye  with  an  in- 
termittent  source  (Bowen  and  Guinness , 1964).  Characteristically,  a 
"jumping11  effect  is  observed  when  the  source  is  small  and  bright;  it  is  due 
to  successive  images  striking  different  points  in  the  retina  during  eye 
movements.  The  basic  remedies  for  flicker  effects  are  to  increase  the 
"on"  phase  of  the  duty  cycle  and  to  speed  up  the  flicker  rate  as  much  as 
may  be  possible.  In  general,  it  should  be  recognized  that  high  brightness e 
small  light  sources  and  intermittency  are  together  the  ingredients  for  un- 
wanted motion  to  be  seen.  ; 

The  material  thus  far  provides  basic  information  concerning 
the  human  visual  function  pertinent  to  simulation  design.  In  the  next  sub- 
section (3.1.3)  a means  for  achieving  an  effective  visual  environment  for 
the  purpose  of  training  is  described.  A technique  is  presented  which  will 
provide  assistance,  for  decisions  on  visual  simulation  equipments  needed 
for  specific  training  applications. 

3.1.3  A Technique  for  Developing  the  Visual  Simulation  Specifications  . 
In  the  development  of  performance  specifications  for  visual  simulation 
equipment  to  be  used  for  training,  the  initial  and  most  important  step  i^ 
the  selection  of  an  engineering  approach  appropriate  to  the  problem.  Al- 
though the  results  of  this  selection  process  are  expressed  in  terms  of  an 
engineering  specification  the  decision  involves  far  more  than  engineering 
considerations  alone.  The  decision  process  i^a  dynamically  interactive 
one  and  should  involve  an  interdisciplinary  team  of  mechanical  or  elec- 
tronic engineer s Human  Factors  Specialists,  Physicists,  computer  pro- 
graming specialists,  and  personnel  experienced  in  the  operational 
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environment  to  be  simulated.  Often,  the  basic  advantage  of  an  interdis- 
ciplinary approach  is  not  properly  exploited  simply  because  the  informa- 
tion which  is  available  is  scattered,  poorly  organized  and  poorly  commu- 
nicated within  the  team,  The  purpose  of  this  subsection  is,  therefore, 
to  describe  and  illustrate  the  use  of  an  analytic  technique  designed  to: 

1)  collate  the  scattered  bits  and  pieces  of  information  about  any  specific 
visual  simulation  problem  as  seen  from  the  standpoints  of  the  various 
team  members,  2)  organize  this  information  in  a systematic  (albeit  arbi- 
trary) manner,  and  3)  display  the  resulting  data  in  a format  which  will 
permit  direct  comparison  of  the  relative  advantages  and  disadvantages  of 
competing  hardware  systems. 

The  technique  described  here  requires  a detailed  knowledge  of 
the  constraints  imposed  on  the  hardware  selection  process  by  the  following 
factors;  a)  the  characteristics  of  the  operational  environment  to  be  simu- 
lated, b)  the  perceptual  requirements  of  the  specific  mission  or  mission 
segment  involved;  and  c)  the  functional  characteristics  of  the  simulation 
hardware  under  consideration.  It  should  be  made  clear  that  this  technique 
does  not  abridge  in  any  way  the  responsibilities  or  the  prerogatives  of  the 
individual  team  members  in  arriving  at  their  joint  decision.  It  should, 
rather,  be  regarded  as  a convenient  method  of  organizing  and  displaying 
the  information  already  available. 

The  ultimate  value  of  this  approach  depends  in  part  on  the  de- 
gree of  active  participation  by  the  individual  team  member's..  Although  the 
basic  technique  can  be  profitably  employed  on  an  individual  basis  the  data 
will  more  accurately  reflect  the  combined  knowledge  of  the  group  if  each 
member  makes  his  unique  contribution  to  the  analysis. 

In  essence,  the  method  consists  of  a logically  based  fractiona- 
tion of  visual  simulation  requirements  (derived  from  a)  ap.d  b)  above)  on  the 
one  hand,  and  simulation  system  characteristics  on  the  other,  into  units  of 
a size  which  will  permit  the  known  facts  about  each  to  be  arrayed  in  a series 
of  simple  decision  matrices  . Thus,  for  each  combination  of  vehicle/mis- 
sion segment  (e . g .,  fixed  wing , low  altitude  navigation),  and  candidate 
simulation  system  (e.g.  , CCTV-Model)  a number  of  matrix  "scores11  can 
be  derived,  each  of  which  represents  for  a given  characteristic,  the  degree 
to  which  the  simulation  system  in  question  meets  the  requirements  imposed 
by  the  specific  environment/ mis sion  to  be  simulated.  When  the  individual 
matrix  scores  are  graphically  displayed  as  a function  of  the  arrayed  simu- 
lation requirements  , a profile  is  obtained  in  which  the  relative  strengths 
and  weaknesses  of  the  simulation  system  under  consideration  are  made 
readily  apparent.  Comparison  of  the  profiles  derived  for  each  of  several 
candidate  simulation  systems  as  a function  of  the  same  set  of  simulation 
requirements  thus  provides  a logical  and  objective  basis  for  the  selection 
of  hardware. 
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Since  most  forms  of  vehicular  control  are  strongly  dependent 
upon  visually  derived  inputs  it  is  apparent  that  the  need  for  visual  simula- 
tion in  training  equipment  spans  the  entire  gamut  of  military  environments. 
While  vehicular  control  training  is  the  primary  source  of  visual  simulation 
requirements  there  are  additional  requirements  derived  from  the  need  to 
provide  training  intactical  interpretation  of  visual  signals  which  are  them- 
selves processed  representations  of  the  real  world,  such  as  radar  and  sonar. 

Although  the  number  and  variety  of  potential  visual  simulation 
problems  is  very  large,  those  associated  with  vehicular  control  can  be 
logically  classified  and  organized  in  terms  of  a)  the  operational  environment 
(e.g.,  air-to-ground,  waterborne  y etc.),  b)  the  type  of  vehicle  (fixed-wing, 
surface  ship,  etc  . ) , and  c)  the  mission  or  mission  segment  (e.g.,  low 
altitude  visual  navigation,  harbor  navigation,  etc.)  which  they  represent. 

The  remainder  consist  of  artificial  sensor  display  problems  and  a few 
special  cases  which  are  distinct  from  but  so  closely  related  to  visual  simu- 
lation that  they  are  often  considered  under  the  same  rubric.  The  classifi- 
cation scheme  adopted  for  the  purposes  of  this  analysis  is  presented  in 
Table  16.  This  particular  classification  while  not  definitive,  has  proved 
useful  because  it  permits  the  total  mission  for  a given  weapon  system  to 
be  broken  down  into  meaningful  segments,  each  of  which  is  well  enough 
defined  to  be  dealt  with  on  an  analytic  basis.  For  example,  a carrier- 
based  attack  aircraft  on  a strike  mission  might  conceivably  involve  several 
of  the  individual  mission  segments  (shown  in  Table  16)  each  of  which  has 
certain  implications  for  the  simulation  hardware  to  be  employed. 

Simulation  of  the  complete  strike  mission  (including  carrier  take- 
off and  landing,  low  altitude  visual  navigation,  target  acquisition,  weapons 
delivery,  etc.)  therefore,  requires  cascading  of  the  analytically  determined 
hardware  constraints  associated  with  each  of  the  mission  segments  in- 
volved. Such  progressive  restriction  of  alternative  solutions  quickly  re- 
veals that  there  is  at  present  no  single  technique  capable  of  providing  a 
satisfactory  solution  to  the  requirement  for  a total  strike  mission  simulator. 
But  more  importantly,  this  technique  can  be  used  to  reveal  the  extent  to 
which  various  mission  segments  (part-tasks)  can  be  combined  in  practical 
and  reasonably  cost-effective  training  equipment. 

The  number  of  engineering  approaches  available  to  solve  the  prob- 
lems defined  by  the  selected  mission  segments  is  limited.  These  approach- 
es are  listed  in  Table  17.  While  this  listing  is  not  inviolate , the  intent  is 
to  include  all  of  the  various  technologies  which  have  or  can  be  used  to  gen- 
erate, store,  transfer  and  display  imagery  for  the  purposes  of  visual 
simulation.  ■’ 

As  previously  stated,  the  successful  application  of  this  technique 
requires  a more  or  less  detailed  knowledge  of  the  relative  strengths  and 
weaknesses  of  the  simulation  hardware  available.  At  the  very  least,  the 
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( l)  Tactical  Formation  Flying 

(2)  ASW  Station  Keeping 


2.  Air-to-  Cround  Environment 
a , Fixed  Wing 

( 1)  Visual  Navigation 

(2)  Visual  Reconnaissance 

(3)  Target  Acquisition 

(4)  Weapons  Delivery 

(a)  Bombs 

(b)  Visually  guided  missiles 
(e)  Gunnery 

JU  Fixed 
2.  Flexible 

(5)  Takeoff/ landing  (fixed  base) 

(a)  Night 

(b)  Day 

(6)  Takeoff/landing  (carrier-based) 

(a)  Night 

(b)  Day 


b.  Rotary  Wing 
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TABLE  16.  MILITARY  VISUAL.  SIMULATION  PROBLEMS  (Continued) 


(1)  Visual  Navigation 

(2)  Visual  Reconnaissance 

(3)  Target  Acquisition 

(4)  Weapons  Delivery 

(a)  Visually  guided  missiles 

(b)  Gunnery  (flexible) 

c.  Remotely  Controlled  (TV  guided)  Missiles 

B.  Surface  Vehicles 

1 . Land  Based  Environment 

a.  Off  Road  Vehicles 

(1)  Tanks/tracked  Vehicles 

(2)  Air  Cushion  Vehicles 

b.  Conventional  Vehicles 

2.  Waterborne  Environment 

a.  Surface(d)  Ship  Control  Problems 

(1)  Harbor  Navigation 

(2)  Ship/ submarine  Mooring 

(3)  Ship/ submarine  Docking 

(4)  Ship-to-Ship  Underway  Replenishment 

b.  Small  Boats  and  Special  Purpose  Waterborne  Vehicles 

(1)  Assault  Boats 

(2)  Hydrofoil 

C.  Subsurface  Vehicles 

1.  Subsurface-to-Surface  Environment  (Periscope  Simulation) 

a.  Approach/ attack 

b.  Reconnaissance 
e.  Navigation 

2.  Subsurface  Environment  (in-water  simulation) 
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TABLE  16.  MILITARY  VISUAL  SIMULATION  PROBLEMS  (Continued) 


|lJ..  Artificial  Sensor  Display  Problems 

j A.  Radar/radar  landmass 

I B . Sonar 

C.  IR 

■ D.  TV-Data  Link 

| E.  ECM 

F.  Compound  types 

i 

i 

i III.  Special  InforniaiiOu  Display  Problems 


A.  Status  and  Large  Area  Tactical  Information  Displays 
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TABLE  17.  BASIC  VISUAL  SIMULATION  SYSTEMS 

1.  Optically  Viewed  Models 

2.  Closed-Circuit  TV  Viewed  Models 

3.  Point  Light  Source  - Transparency 

4.  Open-Loop  Movie 

5.  Servoed  Projections  i 

6.  Semi-Closed  Loop  (VAMP)  Movie 

7.  Computer-Generated  Display 

8.  Flying  Spot  Scanner  - Factored  Transparency 

9.  Laser/ Holographic  Displays 

10.  Hybrid  Systems 
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human  factors  psychologist  must  understand  the  major  advantages  and  dis- 
advantages of  each  of  the  competing  systems  under  consideration.  To  the 
extent  that  this  information  can  be  refined  or  extended  thru  the  participa- 
tion of  a systems  engineer  experienced  in  visual  simulation  (interdisciplin- 
ary team  effort)  the  precision  of  the  resulting  analysis  can  be  improved. 


It  is  not  our  purpose  to  present  an  exhaustive  analysis  and  detailed 
comparison  of  the  specific  characteristics  of  each  of  these  systems.  Rather, 
it  is  assumed  that  the  reader  either  possesses  or  has  available  to  him  the 
background  information  required  to  relate  these  system  characteristics  to 
the  perceptual  requirements  of  the  training  situation.  However,  it  is  de- 
sirable from  the  standpoint  of  orientation  to  review  briefly  the  principal 
features  of  each  basic  system. 

1 .  Optically  viewed  models. 

a.  Typical  application  - periscope  view  simulation. 

b.  Primary  advantages  - high  resolution,  closed  loop 
operation,  realistic  cues  to  depth,  true  colors 
available . 


c.  Primary  disadvantages  - low  brightness,  narrow 
angle  of  view,  difficult  scaling  problems. 

2.  Closed  Circuit  Television  -viewed  models. 

a.  Typical  Applications  - air-to-ground  target  acqui- 
sition, carrier  landing. 

b.  Primary  advantages  - moderately  high  resolution 
at  moderate  brightness  levels,  closed-loop  opera- 
tion, realistic  depth  cues,  electronic  manipulation 
of  imagery  possible  (raster  shrinking,  video  in- 
sertion, selective  blanking,  etc.). 

c.  Primary  disadvantages  - narrow  angle  of  view, 
difficulty  in  achieving  flexible  scaling. 

3 . Point  Light  Source  and  Transparency  Systems . 

a.  Typical  application  - approach  and  landing  simulators. 

b.  Primary  advantages  - extremely  wide  angle  field  of 
view  available,  closed-loop  operation,  full  color  easy 
to  provide . 
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c.  Primary  disadvantages  - low  resolution  and  low 
brightness,  poor  depth  cues,  difficult  scaling 
problems  . 

4 . Open-L.oop  Movie 

a.  Typical  application  - low  altitude  navigation  and 
target  acquisition  simulators. 

b.  Principal  advantages  - high-resolution,  high  brightness, 
full  color,  extremely  wide  angle  of  view,  good  depth 
cues  available,  scaling  inherent  in  film. 

c.  Principal  disadvantages  - open-loop  operation, 
difficulty  in  acquiring  adequately  controlled  imagery. 

5.  Servoed  projections 


a.  Typical  application  - Redeye  Moving  Target  simulator. 

b.  Primary  advantages  - high  resolution,  high  brightness, 
extremely  wide  angle  field. 

c.  Primary  disadvantages  - reversed  contrast  of  targets 
(target  appears  as  bright  image  against  dark  back- 
ground), open-loop  operation. 

6.  Semi-Closed  Loop  (VAMP)  Movie. 


a.  Typical  application  - takeoff  and  landing  simulators. 

b.  Principal  advantages  - high  resolution,  high  bright- 
ness, full  color,  good  depth  cues,  scaling  inherent 
in  film,  semi-closed  loop  capability  permits  limited 
maneuvering  within  a narrow  envelope. 

c.  Principal  disadvantages  - deliberately  introduced 
optical  distortion,  difficulty  of  acquiring  proper 
imagery . 

7 .  Computer  Generated  Displays. 

a.  Typical  application  - takeoff  and  landing,  air-to-air 
combat  maneuvering,  ship  (locking  and  station  keep- 
ing, etc. 
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b.  Principal  advantages  - closed-loop  operation,  scaling 
inherent  in  programing,  independent  motion  available 
within  visual  field,  resolution  and  color  elements  can 
be  differentially  apportioned  according  to  areas  of 
interest,  entire  simulation  problem  can  be  changed 
or  the  elements  within  a problem  updated  by  pro- 
graming. 

c.  Principal  disadvantages  - relatively  high  system  and 
software  costs,  imagery  consists  of  line  drawings  or 
discontinuous  cartoon-like  figures. 

8 * Flying  Spot  Scanner  - Transparency  Systems. 

a.  Typical  application  - radar  landmass  simulators. 

b.  Principal  advantages  - closed-loop  operation,  method 
of  reading  out  information  (density  along  narrow  beams 
sequentially  scanned  by  a small  spot)  produces  very 
good  analogue  of  a scanning  radar  beam. 

c.  Principal  disadvantages  - resolution  poor  at  low 
simulated  altitudes,  high  cost  of  producing  properly 
coded  transparencies  at  scale  factors  required. 

9.  Laser /Holographic  Displays. 

a.  Typical  applications  - high  brightness,  large  screen 
tactical  situation  displays,  carrier  landing  simulators. 

b.  Principal  advantages  - very  high  brightnes s , multiple 
colors,  and  variable  persistance  of  photochromically 
recorded  laser  images  makes  them  ideal  for  large 
screen  status  displays.  Three  dimensional  quality 

of  holographic  imagery  is  valuable  in  simulating 
carrier  landing,  ship  docking,  etc. 

c.  Principal  disadvantages  - narrow  angle  available 
with  Holograms  , monochrome  images,  experimental 
nature  of  current  technology. 


10.  Hybrid  Systems  (employing  a combination  of  two  or  more 
of  the  basic  approaches  outlined  above). 


a.  Typical  application  - visual  mission,  for  example, 
simulator  for  the  A-7D  WST.  This  proposed  system 
was  to  provide  visual  simulation  for  takeoff,  low 
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altitude  navigation,  target  acquisition,  weapon  delivery 
and  landing,  by  incorporating  the  basic  characteristics 
of  a transparency  based  terrain  model,  and  a semi- 
closed  loop  (VAMP-type)  projection  system. 

b.  Principal  advantages  - high  brightness,  high  resolution, 
full  color,  semi-closed  loop  capability. 

c.  Principal  disadvantages  - narrow  angle  of  view,  cum- 
bersome solution  to  scaling  problems  (necessity  for 
repeated  alternation  of  imagery  stored  in  film  cassetts), 
at  least  seven  separate  sources  of  distortion  whose 
effects  cumulate  in  unpredictable  ways. 

It  will  be  apparent  to  the  experienced  reader  that  the  simplified 
summaries  presented  above  could  be  questioned  in  specific  instances.  For 
example,  it  could  be  argued  that  TV-based  systems  are  not  necessarily 
limited  to  narrow  fields  of  view.  However,  in  order  to  achieve  a wide 
field  one  must  sacrifice  both  resolution  and  brightness  or  accept  the  prob- 
lems inherent  in  achieving  and  maintaining  image  registration  in  multiple 
adjacent  displays.  Similarly,  it  could  be  pointed  out  that  the  narrow  angle 
of  view  which  is  characteristic  of  optically  viewed  systems  is  not  a serious 
disadvantage  in  periscope  view  simulations.  In  this  case  the  angle  of  view 
is  a disadvantage  of  the  general  system  type,  though  not  in  the  application 
cited. 


It  should  be  noted  that  in  listing  the  relative  advantages  of  the 
various  systems,  two  different  types  of  features  are  cited.  Some  features 
such  as  resolution,  brightness,  distortion  and  displayed  angle  of  view  are 
usually  considered  basic  parameters  of  any  visual  simulation  sy stem.  Other 
features  such  as  the  ability  to  provide  independent  motion  of  targets  within 
the  visual  scene,  the  relative  difficulty  in  making  changes  in  the  stored 
imagery  and  whether  or  not  the  image  storage  medium  is  capable  of  provid- 
ing adequate  cues  to  depth  are  not  ordinarily  thought  of  as  system  param- 
eters . Although  the  distinction  between  them  is  somewhat  arbitrary  it  is 
a significant  one  since  the  most  important  aspect  of  this  analytic  technique 
is  that  it  deals  explicitly  with  those  factors  which  are  important  in  the 
selection  of  simulation  hardware  but  which  are  not  recognized  as  quantita- 
tive system  parameters.  These  are  the  somewhat  nebulous,  difficult  to 
quantify  but  highly  important  considerations  which  are  usually  a product  of 
each  individual  team  member’s  experience.  As  such  they  often  play  a de- 
cisive but  implicit  role  in  the  decision-making  process.  The  basic  problem 
with  such  expe rientially- de pendent  decisions  lies  in  the  individual  variations 
with  regard  to  the  implicit  definitions  of  these  factors  and  in  the  relative 
weights  assigned  to  each.  By  identifying  and  explicitly  defining  as  many  of 
the  system  characteristics  which  bear  on  the  hardware  selection  process  as 
possible  this  analysis  will  enable  the  individual  or  team  to  arrive  at  their 
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decisions  on  a more  logical  basis.  The  system  characteristics  to  be  con- 
sidered for  the  purpose  of  this  analysis  are  listed  and,  where  necessary, 
discussed  below. 

3 . 1 . 3 . 1 Visual  System  Characteristics  . 

3. 1.3.1.  1 Scaling.  This  characteristic  is  concerned  with  the  degree 

to  which  the  medium  used  to  store  visual  information  is  able  to  provide  a 
simulated  maneuvering  area  sufficient  to  meet  the  requirements  without 
the  necessity  of  changing  scale  factors.  For  example,  for  the  purpose  of 
low  altitude  visual  navigation  training,  the  inflexible  scaling  of  the  CCTV  - 
model  system  is  a severe  disadvantage  because  of  the  conflicting  require- 
ments for  high  resolution  and  large  area  coverage.  On  the  other  hand,  for 
a simulation  problem  such  as  ship -to- ship --under way  replenishment  involv- 
ing the  approach  and  station  keeping  of  two  large  objects  on  an  essentially 
flat,  featureless  and  infinite  plane  (the  open  sea)  this  characteristic  does 
not  constitute  a disadvantage  for  the  CCTV  system. 

3. 1.3. 1.2  Resolution.  This  is  one  of  the  main  physical  parameters  of 
any  system.  For  the  purpose  of  this  analysis,  however,  the  specific  values 
are  less  important  than  the  gross  categorization  of  systems  in  terms  of 
whether  their  resolution  is  representative  of  optical  quality,  photographic 
quality  or  TV  quality. 

3. 1.3. 1.3  Detail  Requirements.  This  characteristic  refers  to  the 
amount  and  type  of  detail  which  must  be  available  in  the  visual  scene  to 
meet  the  mission  requirements.  It  therefore  defines  the  problems  to  be 
solved  in  the  selection  and  encoding  of  visual  information  for  storage.  As 
one  might  expect,  the  importance  of  this  characteristic  varies  widely  de- 
pending on  the  image  storage  medium  involved.  For  example,  it  is  rela- 
tively unimportant  in  most  photographically  based  systems,  becomes  a 
significant  factor  in  the  design  of  CCTV-model  systems  and  is  a critical 
aspect  of  computer-generated  systems. 

3.  1.3. 1.4  Color  Capability.  This  characteristic  refers  to  the  fidelity 

of  color  reproduction  of  the  system.  For  the  purpose  of  this  analysis  only 
gross  categorization  is  required.  Systems  can  be  classified  simply  on  the 
basis  of  whether  they  are  capable  of  producing  black  and  white  only,  partial 
color  as  in  systems  based  on  2-color  mixtures,  or  full  color  as  in  systems 
based  on  3-color  mixtures  or  optically  viewed  moxlels  . 

3. 1.3. 1.5  Absolute  Brightness.  This  is  a basic  parameter  of  all  simu- 

lation systems.  For  obvious  reasons  low  absolute  brightness  is  frequently 
a serious  disadvantage  of  systems  designed  to  provide  extremely  wide  fields 
of  view. 
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3. 1.3.  1.6  Contrast  Range . This  is  a self-explanatory  system  charac- 

teristic. In  general,  however,  if  a system  delivers  adequate  brightness  the 
contrast  range  will  also  be  adequate. 

3.  1.3. 1.7  Distortion  Tolerance.  In  the  context  of  this  analysis  this 

characteristic  refers  to  the  perceptual  acceptability  of  simulation  imagery 
which  has  been  purposely  distorted  by  the  system,  as  in  the  VAMP-type 
semi-closed  loop  movies. 

3.  1.3.  1.8  Flicker  Tolerance.  This  is  a basic  factor  in  both  TV  and 

motion  picture  systems.  Although  it  does  not  usually  present  a problem, 
it  can  be  a serious  disadvantage  in  some  applications  of  extremely  wide 
angle  motion  picture  systems. 

3.  1. 3. 1.9  Accommodation  to  Infinity.  In  the  context  of  this  analysis 

this  characteristic  refers  to  the  relative  ease  or  difficulty  of  displaying  the 
obtained  imagery  in  such  a way  that  it  appears  at  infinity.  For  example, 
some  systems  lend  themselves  to  the  use  of  relatively  compact  virtual 
image  displays,  while  others  require  that  the  image  plane  be  physically 
remote  from  the  viewer,  therefore  resulting  in  a large  and  expensive  in- 
stallation to  achieve  the  same  effect. 

3.1.3.1.10  Relative  Motion.  This  factor  is  concerned  with  the  angular 
velocities  expected  on  the  basis  of  mission  requirements  and  with  their 
anticipated  effects  on  various  system  parameters  such  as  image  smear, 
stroboscopic  effects,  and  the  detail  requirements  of  the  storage  medium. 

3.1.3.1.11  Independent  Closed- Loop  Motion.  This  characteristic  refers 
to  the  relative  ease  or  difficulty  in  providing  independent  closed-loop  motion 
of  targets  within  the  simulated  visual  environment.  For  some  missions 
such  as  air-to-air  combat  maneuvering  this  capability  is  essential.  A 
mission  segment  such  as  weapon  delivery  (including  all  types  of  missiles, 
fixed  and  flexible  gunnery),  which  requires  the  provision  of  visual  ground- 
effects,  constitutes  a special  case  in  that  the  position  of  impact  is  deter- 
mined by  ballistic  computation  (i.e.  , the  control  loop  is  closed  mechanically 
or  electronically  rather  than  manually).  Simulation  systems  differ  widely 

in  their  ability  to  provide  this  feature.  For  example , while  it  is  possible 
to  utilize  a CC TV-model  system  for  air  combat  simulation,  the  solution  is 
necessarijly  large  and  complex.  Computer-generated  displays,  however,  are 
well  suited  to  such  problems  and,  in  addition,  can  provide  any  type  of 
weapons  effects  desired,  including  even  tracer  paths. 

3.  1.3.1.  12  Coupled  Motion.  This  factor  is  concerned  with  the  feasibility 
of  coupling  a motion  platform  with  the  visual  system  under  consideration  to 
provide  the  integrated  sensory  cues  required  for  vehicle  control  training. 
Mission  considerations  dictate  both  the  requirement  for,  and  the  degree  of 
coupling.  For  example,  for  stable  and  slow-moving  vehicles  such  as  ships 
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itovul-r*?*!  hathi&r  navigation  and  40  docking,  visual  ropfesentatiuo  of  Ike 
motion  toe*  ia  soffit  lent  in  ilicUa  ft5  iflulatieu  of  an  assault.  boa  1 landing, 
too  we  vet-.  requires  precise  (tfeopJihe  of  the  two  Syilew#  because  the  vehicle 
(OMfol  task  is  * rititally  dependent  open  line  f»(iUion<it<ift  b«(woen  the  visual 
and  motion  cues* 

1.  1 , 3. 1 . 1 3 Closed*  v«,  Open-Loop  Operation*.  THic  bane  parameter 
iff  determined  toy  whether  nr  not  control  actions  are  fed  back  into  the  system. 
In  open-loop  systems  there  is  no  control  feedback  and  the  resultant  imagery 
i«  fitted  or  programed  as  in  most  vehicle  control  simulations  or  mechanically 
as  in  the  si  mulatto  > of  weapons  effects  v ...tch  was  mentioned  earlier.  The 
critical  importance  of  this  parameter  in  many  training  situations  is  due  to 
the  fact  that  closed-loop  operation  demands  active  participation  of  the  trainee 
in  accomplishing  the  control  task  and  provides  feedback  to  him  concerning 
degree  of  success.  Special  techniques  are  available  to  achieve  these  ends 
in  certain  types  of  open -loop  simulation  but  in  general  they  are  more  lim- 
ited in  their  application  and  therefore  less  satisfactory. 

< .1.3.1.14  Angle  of  View.  This  Is  a critical  parameter  in  the  design  of 
simulators  for  many  training  purposes.  Other  things  being  equal,  it  is 
desirable  to  provide  the  widest  angle  of  view  achievable.  Some  mission 
requirements,  however,  are  more  stringent  than  others  in  this  respect. 

For  example,  for  air-to-ground  weapons  delivery,  a wide  angle  of  view  is 
desirable  but  not  essential  since  bomb,  missile  and  fixed  gunnery  targets 
will  generally  be  close  to  the  flight  path.  On  the  other  hand,  for  simulation 
of  the  low  altitude  navigation  problem  visual  information  from  the  extreme 
periphery  is  essential  and  therefore  an  extremely  wide  angle  display  is 
required. 


3.1.3.1.13  Depth  of  Field.  This  Is  a basic  parameter  which  does  not 
usually  constitute  a problem  for  most  system/ mission  combinations,  since 
in  most  cases  the  actual  distance  from  optical  probe  to  ares  of  interest  is 
sufficiently  great  that  a normal  optical  system  stopped  down  to  a small 
aperture  will  yield  adequate  depth.  A very  small  lens  aperture  of  course, 
creates  other  problems  such  as  the  necessity  to  either  increase  the  illu- 
mination level  or  the  sensitivity  of  the  recording  device.  There  are, 
however*  mission  specific  situations  in  which  depth  of  field  does  present 
a serious  system  limitation  independent  of  both  brightness  and  sensitivity, 
e.g. , when  an  optical  probe  is  required  to  come  very  close  to  a surface 
such  as  in  a CCTV-model  simulation  of  carrier  takeoff  and/or  landing. 

3.1.3.1.16  Ease  of  Change.  This  factor  refers  to  the  relative  ease  of 
making  changes  in  the  visual  information  which  is  already  coded  and  stored 
In  the  system.  It  is,  of  course,  related  to,  but  not  identical  with  the  scal- 
ing factor  discussed  earlier.  While  flexible  scaling  refers  to  the  complete 
replacement  of  one  stored  visual  environment  by  another  having  a different 
scale  factor,  ease  of  change  refers  to  the  relative  simplicity  of  updating  an 
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rxisiing  imaf^  store  by  making  small  loealiited  change*.  This  characteristic 
can  be  illustrated  by  comparing  the  photographic  (factored  transparency)  and 
the  computer-generated  approaches  to  radar  land  mass  simulation.  Factored 
transparency  systems  which  represent  geographic  and  cultural  features  as  a 
function  of  elevation  in  one  transparency  and  as  a function  of  radar  reflectivity 
in  another,  cannot  be  updated  when,  for  example,  radar  targets  such  as 
bridges,  highways,  dams,  reservoirs,  etc.,  are  created  or  destroyed.  It 
is,  of  course,  apparent  that  computer-generated  imagery  can  be  updated 
with  ease  by  appropriate  programing  changes. 

3. 1 . 3. 1 . 17  Monocular  Movement  Parallax.  This  is  one  of  several  factors 
concerned  with  the  cues  which  govern  the  perception  of  apparent  depth  in  the 
imagery  delivered  by  a system.  Monocular  movement  parallax  refers  to 
whether  or  not  objects  in  the  background  of  a scene  appear  to  move  in  rela- 
tion to  the  foreground,  as  a function  of  relative  motion  between  the  observer 
and  the  target  area.  Since  both  the  direction  and  velocity  of  this  apparent 
motion  constitute  cues  to  the  depth  of  an  object  i"\  the  real  world,  this  factor 
is  a powerful  determiner  of  the  corresponding  illusion  in  the  simulated 
environment.  Generally,  monocular  movement  parallax  is  based  upon  three- 
dimensional  inputs  since  it  depends  upon  constantly  changing  aspect  angles 
between  the  recording  device  and  objects  in  the  visual  field.  Although  a 
regularly  changing  target  aspect  angle  is  a necessary  condition  for  deriving 
the  movement  parallax  depth  cue,  it  is  not  in  itself  sufficient;  the  target 
aspect  angles  must  be  changing  at  or  above  some  threshold  value.  The 
practical  significance  of  these  considerations  is  that,  taken  together,  they 
define  a range  limitation  for  some  mission.*  (e.g.,  air-to-ground  target 
acquisition,  remotely  controlled  (TV)  guided  missiles)  beyond  which  depth 
cues  based  on  this  principle  will  not  be  effective. 


3.1.3.1.18  Interposition.  This,  like  monocular  movement  parallax,  is 
another  of  the  basic  cues  to  depth.  However,  unlike  the  former,  the  illusion 
of  depth  is  not  dependent  upon  motion  but  upon  the  individual's  learned  re- 
sponse to  certain  characteristics  of  identifiable  patterns.  Through  experi- 
ence, an  individual  learns  to  interpret  certain  shapes  as  representative  of 
certain  classes  of  objects.  When  the  expected  image  of  a man  interrupts 
or  cuts  off  a portion  of  the  image  to  be  expected  of  a tree,  the  man  is  inter- 
preted as  b' Ing  in  front  of  (closer  to  the  observer  than)  the  tree.  Provided 
that  the  geometric  relationships  between  the  relative  size  of  known  objects 
is  preserved,  within  very  broad  limits,  interposition  is  a simple  but  very 
powerful  cue  to  depth.  Most  simulation  systems  which  are  based  upon  three- 
dimensional  inputs  (optical  or  TV-viewed  scale  model,  movies  based  on 
either  the  real  world  or  models  of  it)  provide  this  cue  automatically.  Arti- 
ficially calculated  visual  environments  (e.g.  , computer-generated  imagery, 
animated  motion  pictures)  can  also  provide  these  cues  but  at  some  cost  in 
programing  effort.  In  some  cases,  however  (e.g.  , point  light  source  - 
transparency  systems),  the  inability  to  provide  this  cue  is  a substantial  dis- 
advantage of  the  system. 
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3.  1.3.1  19  Phy?  »1  Size  . This  Is  an  obviously  important  aspect  of  any 
system  In  that  It  Is  . *ely  related  to  overall  cost.  In  the  context  of  this 
analysis  only  gross  categorization  is  necessary.  These  categories  are 
defined  as  follows:  (1)  small- -desk-lop  to  small  classroom  size,  (2)  med- 
ium- -a  space  requiring  special  preparation  such  as  an  unusually  high 
ceiling,  a false  floor,  special  cooling  arrangements,  etc.,  and  (3)  large-- 
requiring  a special  building,  exceptionally  large  room,  or  an  unusually 
extensive  modification  of  an  existing  building. 

3.1.3.1.20  Power  Consumption.  This  is  another  cost-reiaied  parameter 
which  both  reflects  and  is  reflected  in  the  size,  weight,  illumination  and 
cooling  requirements  of  the  system.  For  the  purpose  of  this  analysis  the 
following  categories  are  defined: 

(1) small--up  to  10  KW 

(2)  medium-- 10  to  50  KW 

(3) large--over  50  KW 

3.1.3.1.21  Original  Hardware  Cost.  For  the  purpose  of  this  analysis 
the  following  categories  are  defined: 

Low- -up  to  250K 
Medium- -250 K to  1.5M 
High-  -above  1 . 5M 

3.1.3.1.22  Original  Software  Costs.  For  the  purpose  of  this  analysis, 
original  software  costs  are  classified  as  follows: 

Low 

Medium 

High 

3.1.3.1.23  Maintenance  Costs.  This  factor  includes  both  hardware  (parts 
replacement  and  equipment  repair)  and  software  (programing)  support  re- 
quired for  adequate  availability.  For  the  purpose  oi  this  analysis  mainten- 
ance costs  are  classified  as  follows: 

Low 

Medium 

High 

On  the  basis  of  the  foregoing  discussion,  it  is  apparent  that 
the  importance  to  be  attached  to  any  individual  system  characteristic  varies 
widely  depending  on  both  the  mission  and  the  simulation  system  involved. 
Specifically,  it  should  be  noted  that: 
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a.  While  all  of  the  23  characteristics  cited  are  relevant  to 
some  combination  of  mission  segment  and  simulation  technology,  few,  if 
any,  have  relevance  for  the  entire  range  of  such  possible  combinations. 


b.  A given  system  parameter,  such  as  resolution  (which  is 
a quantitatively  measurable  aspect  of  any  display  system),  is  a significant 
characteristic  only  if  it  is  substantially  greater  than, or  less  than,  that  re- 
quired on  the  basis  of  mission  considerations.  For  example,  if  the  avail- 
able system  resolution  exceeds  the  resolution  required  to  simulate  a given 
operational  environment,  the  proposed  system  represents  a significantly 
uoverengineeredu  (and  therefore  overpriced)  solution  in  respect  to  this 
particular  parameter.  Similarly,  wherever  the  system  capability  is  sub- 
stantially less  than  the  requirement,  the  parameter  in  question  constitutes 
a significant  disadvantage  of  the  system  for  that  specific  application. 


c.  While  only  those  systems  characteristics  for  which  re- 
quirements and  capabilities  are  for  some  reason  unequal  are  to  be  consid- 
ered significant,  it  is  apparent  that  not  all  significant  characteristics  are 
important  from  the  practical  standpoint.  This  is  because  the  nature  of  our 
primary  interest  necessarily  emphasizes  the  disadvantages  or  shortcomings 
of  competing  technologies  rather  than  their  advantages.  For  example,  in 
our  context,  a parameter  which  is  identified  as  a system  advantage  for  a 
given  application  usually  requires  no  further  consideration.  Conversely,  a 
characteristic  identified  as  a system  disadvantage  must  be  further  consid- 
ered in  terms  of  its  criticality,  i.e.  , a system  disadvantage  may  be  either 
a critical  or  a non-critical  factor  in  the  ultimate  hardware  selection  process. 
The  designation  of  individual  system  characteristics  as  important  or  critical 
in  achieving  a specified  simulation  goal  is  a judgment  which  requires  con- 
siderable knowledge  and  experience  on  the  part  of  the  individual  or  team 
involved.  These  judgments  constitute  an  important  step  in  the  overall  de- 
cision process  in  that  they  identify  those  difficulties  which  must  be  over- 
come by  each  of  the  candidate  technologies  in  order  to  achieve  the  desired 
objective.  In  terms  of  this  analysis,  a critical  system  deficiency  for  any 
specified  application  is  defined  as  one  which  can  be  circumvented  only  at 
considerable  cost  to  the  system,  in  terms  of  money,  complexity,  develop- 
ment time,  maintainability  and  engineering  risk. 


3.  1.3.2  Application  of  the  Technique.  The  application  of  this  technique 
to  an  actual  visual  simulation  problem  is  illustrated  in  the  remainder  of  this 
subsection  with  an  analysis  Of  three  candidate  simulation  technologies  as  a 
function  of  the  same  set  of  simulation  requirements.  The  low  altitude  visual 
navigation  problem  (see  Table  18)  is  selected  because  of  the  stringent  re- 
quirements it  imposes  on  the  hardware  solution  with  respect  to  several 
important  system  characteristics,  viz.,  angle  of  view,  scaling,  resolution, 
depth  cues,  and  the  desirability  of  closed-loop  operation.  This  mission 
segment  will  be  analyzed  in  terms  of  the  characteristics  of  1)  CCTV , 

2)  open-loop  movie,  and  3)  computer- generated  simulation  systems.  The 
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analysis  will  result  in  three  separate  system  characteristic  profiles  on  the 
basis  of  which  the  relative  strengths  and  weaknesses  of  each  of  the  candi- 
date systems  can  be  compared. 

In  addition,  the  effect  of  individual  vs.  team  participation  in  the 
analysis  will  be  demonstrated  by  presenting  two  separate  profiles  derived 
from  the  same  basic  information.  That  is,  for  a single  mission/ system 
combination,  two  independently  derived  profiles  will  be  presented- -one 
showing  the  results  of  an  analysis  from  the  standpoint  of  a human  factors 
specialist,  the  other  based  on  the  combined  or  "team11  judgments  of  a human 
factors  specialist  and  a systems  engineer.  Examination  of  the  similarities 
and  differences  in  the  profiles  will  illustrate  the  refinement  and  increased 
precision  of  analysis  which  can  be  achieved  by  integrating  the  complement- 
ary skills  and  knowledges  of  the  individual  participants  into  a team  effort. 


The  format  adopted  for  the  multiple  decision  matrices  is  illus- 
trated in  Figure  22.  Except  for  a few  cases,  which  will  be  noted  below,  all 
matrices  are  identical,  consisting  of  nine  cells  in  a 3x3  block  arrangement. 
There  is  an  individual  matrix  for  each  of  the  23  system  characteristics 
(e.g.  , scaling,  resolution)  previously  discussed.  Each  of  these  general 
system  characteristics  is  evaluated  in  terms  of  the  mission-derived  simu- 
lation requirements  on  the  vertical  axis,  and  the  specific  system  character- 
istics available  on  the  horizontal  axis.  Each  cell  of  the  matrix  is  pre- 
numbered with  a “value11  ranging  from  -2  to  +2.  The  specific  digits  in 
themselves  have  no  quantitative  significance --they  could  be  letters  or  arbi- 
trary symbols.  (It  is,  of  course,  a possibility-- with  further  development-- 
that  these  values  could  be  assigned  quantitative  significance,  if  appropriate 
empirical  weightings  were  established  for  each  system  characteristic.) 

The  primary  feature  of  each  matrix  is  that  the  digits  are  arranged  so  that 
a ''O’1  score  is  obtained  whenever,  in  the  judgment  of  the  individual  or  the 
team  making  the  analysis,  the  simulation  requirement  as  determined  by  the 
nature  of  the  mission  is  met  by  the  specific  characteristic  of  the  system 
under  consideration.  Matrix  cells  containing  negative  numbers  will  be  ob- 
tained when  in  the  judgment  of  the  analyst(s)  the  specific  system  character- 
istic is  not  adequate  to  meet  the  requirement.  Similarly,  a positively 
numbered  cell  indicates  an  unneeded  or  excess  capability  of  the  system  in 
question  for  the  purpose  of  achieving  the  defined  objective.  It  should  be 
noted  that,  since  the  format  of  the  matrices  is  essentially  uniform  through- 
out, the  specific  judgments  required  for  each  of  the  requirement/capability 
combinations  va-ry  somewhat.  For  example,  some  matrices  are  simply 
labeled  high,  medium  and  low  on  each  of  the  axes  (viz.  , detail  requirement, 
absolute  brightness,  contrast  range,  ease  of  change).  In  these  cases, 
which  tend  to  be  associated  with  basic  system  parameters,  the  judgments 
to  be  made  are,  (a)  what  level  of  the  specific  characteristic  is  required  on 
the  basis  of  the  known  facts  about  the  mission,  and  (b)  what  is  the  gross 
ability  of  the  system  in  question  with  regard  to  the  specified  characteristic. 
In  some  cases  (e.g.  , resolution,  color,  relative  motion,  angle  of  view. 
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TYPE  OF  VEHICLE  & GENERAL  MISSION: 

SPECIFIC  MISSION  SEGMENT: 
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Figure22.  Format  for  the  Multiple  Decision  Matrices 
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VISUAL  SIMULATION 
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VISUAL  SIMULATION  SYSTEM  REQUIREMENTS 
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etc.)  the  same  basic  judgments  are  required,  but  for  convenience,  the  axes 
are  labeled  in  terms  of  the  characteristic  itself.  Thus,  angle  of  view  is 
labeled  in  terms  of  wide,  medium  and  narrow;  color  in  terms  of  full,  partial 
or  absent. 


Another  type  of  judgment  is  required  for  such  qualities  or  sys- 
tem characteristics  as  independent  motion,  flexible  scaling,  accommodation 
to  infinity,  monocular  movement  parallax  and  interposition.  In  these  cases, 
the  pertinent  judgments  are;  (a)  whether  the  characteristic  in  question  is 
required,  desirable  or  unnecessary  in  order  to  achieve  the  simulation  ob- 
jective as  defined  by  the  mission,  and  (b)  whether  the  characteristic  con- 
stitutes a primary  advantage,  a secondary  advantage,  or  a primary  disad- 
vantage of  the  system  under  consideration. 

Finally,  with  respect  to  several  cost-related  factors  such  as 
power  consumption,  maintenance  costs,  etc.  , the  usual  nine  cell  matrix  is 
reduced  to  three.  This  is  because  by  definition,  only  low  cost,  low  mainte- 
nance, low  power  consumption,  etc.  , are  required  (i.e,  , desired). 


The  format  of  the  system  characteristic  profiles  derived  from 
the  multiple  matrix  scores  is  illustrated  in  Figure  23.  The  23-system 
characteristics  selected  for  analysis  are  arrayed  along  the  horizontal  axis 
and  individual  matrix  scores  are  plotted  vertically.  When  the  individual 
points  are  connected,  a •'profile11  is  formed  which  graphically  portrays  the 
ability  of  the  simulation  system  under  consideration  to  meet  the  simulation 
requirements  of  the  selected  mission.  Though  a given  profile  is  concerned 
with  the  "fit"  of  only  one  system/mis sion  combination,  successive  analyses 
can  be  made  for  each  of  several  candidate  systems  in  order  to  provide  a 
basis  for  comparing  alternative  engineering  solutions. 

As  mentioned  previously  the  analytic  procedure  outlined  here  is 
not  a substitute  for  the  team's  decision-making  process  in  selecting  an  en- 
gineering approach  to  solve  a given  simulation  problem.  It  does,  however, 
insure  that  competitive  (but  often  vastly  different)  technologies  are  evaluated 
logically  and  systematically  on  the  same  bases. 


The  results  of  the  matrix  analyses  of  the  three  candidate  simu- 
lation systems  with  respect  to  the  simulation  requirements  of  the  low  altitude 
visual  navigation  mission  are  summarized  next.  Each  of  the  th^-ee  profiles 
shown  (CCTV  model,  open-loop  movie  , computer- generated  imagery)  is 
based  on  the  judgments  of  an  individual  evaluator  (human  factors  specialist). 
For  the  low  altitude  visual  navigation  problem  three  system  characteristics 
were  designated  as  critical  factors.  These  are,  in  the  order  of  their  pre- 
sumed importance;  angle  of  view,  programed  vs.  unprogramed,  and  scaling. 


In  order  co  fully  understand  the  derivation  and  use  of  the  infor- 
mation presented  in  the  matrix  profiles,  it  is  first  necessary  to  understand 
the  visual  simulation  requirements  of  the  low  altitude  visual  navigation. 
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TYPE  OF  VEHICLE  & GENERAL  MISSION: 

SPECIFIC  MISSION  SEGMENT: 
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Figure  23.  Format  of  System  Characteristics  Profile 
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mission.  As  previously  mentioned,  the  visual  simulation  requirements  are 
derived  from  two  sources,  i.e.  , the  characteristics  of  the  operational  en- 
vironment and  the  perceptual  requirements  of  the  specific  mission  segment 
involved.  Consideration  of  the  above  factors  as  they  affect  the  simulation 
requirements  of  this  mission  reveals  that: 

a.  An  extremely  wide  angle  of  view  is  essential  since  many  of 
the  visual  cues  necessary  for  proper  geographic  orientation  at  low  altitude 
are  derived  from  the  peripheral  field. 

b.  An  unprogramed  (closed-loop)  capability  is  highly  desirable 
since  the  trainee  must  be  able  to  exercise  active  control  of  his  flight  path. 
Whether  or  not  this  feature  is  essential  (as  opposed  to  desirable)  is  a matter 
of  interpretation.  It  can  be  argued  that  since  the  navigation  problem  in- 
volves only  minor  deviations  from  a prescribed  path  down  a defined  series 
of  terrain  corridors,  only  a semi-programed  capability  is  required. 

c.  Some  form  of  flexible  scaling  is  essential  because  the  nature 
of  the  low  altitude  navigation  mission  requires  large  area  coverage  at  fairly 
high  resolution. 

d.  The  ability  to  provide  adequate  depth  cues  is  essential  be- 
cause the  shallow  look  down  angles,  relatively  short  slant  ranges,  and  high 
angular  velocities  characteristic  of  low  altitude  flight  produce  many  opera- 
tionally important  cues  which  are  based  on  interposition  and  on  monocular 
movement  parallax  (e.g.  , tar  get/ terrain  masking -unmasking)  . In  this 
particular  analysis,  however,  neither  monocular  movement  parallax  nor 
interposition  were  designated  as  critical  factors  because  all  three  candi- 
date systems  are  capable  of  meeting  the  requirement.  Thus,  although 
these  characteristics  are  essential  to  a successful  simulation  ox  the  low 
altitude  navigation  mission,  it  would  serve  no  purpose  to  designate  them 
as  critical  factors  since  they  would  not  differentiate  between  possible  en- 
gineering solutions. 

In  addition  to  the  above,  it  should  be  noted  that  the  requirement 
for  visual  information  in  the  extreme  peripheral  field  has  implications  for 
several  other  system  characteristics.  Thus,  in  an  extremely  wide  angle 
display  the  absolute  brightness,  flicker  tolerance,  accommodation  to 
infinity  and  relative  motion  characteristics  take  on  added  importance  and, 
in  fact,  can  become  critical. 

3. 1.3.2. 1 Comparison  of  Systems  Characteristics  . A comparison  of 

the  three  system  characteristic  profiles  shown  in  Figures  24,  25  and  26 
reveals  that  for  simulation  of  the  low  altitude  visual  navigation  mission: 

a.  The  CCTV-model  system  has  two  major  disadvantages  , 
viz.  , narrow  angle  of  view  and  difficult  scaling  problems  . In  addition , it 
requires  a relatively  large  and  costly  installation.  The  primary  advantage 
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Figure  25.  System  Characteristic  Profile  for  the  Open  Loop 
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of  this  type  of  approach  is  in  its  ability  to  provide  unprogramed,  or  closed- 
loop  operation.  The  profile  also  shows  that  resolution  is  marginal  and 
there  is  an  appreciable  susceptibility  to  image  smear.  On  the  other  hand, 
color,  brightness,  contrast,  distortion,  depth  cues  and  original  costs  are 
considered  adequate. 

b.  The  wide  angle  open-loop  movie  system  has  (as  its  name 
implies)  one  major  disadvantage,  open-loop  operation,  and  one  major 
advantage,  the  ability  to  provide  extremely  wide  angle  imagery.  Flexible 
scaling  which  presents  a major  problem  for  the  CCTY-based  system  is  an 
inherent  property  of  the  motion  pictur e-based  system. 

Resolution,  color,  distortion  and  depth  cues  were  all 
judged  to  be  adequate.  There  are,  however,  some  problems  with  strobo- 
scopic effects  and  image  smear  in  the  extreme  periphery.  Size,  hardware 
and  maintenance  costs  are  more  or  less  moderate,  but  the  cost  of  acquiring 
properly  controlled  imagery  (software)  is  high.  It  should  be  noted  here  that 
care  must  be  exercised  in  the  interpretation  of  any  matrix  score  shown  on  a 
profile,  since  any  given  score  is  the  result  of  the  interaction  of  simulation 
requirements  with  system  characteristics.  Thus,  in  Figure  24  both  the 
independent  motion  characteristic  and  the  ease  of  change  characteristic 
are  plotted  as  ,T0Ms,  meaning  that  the  simulation  requirement  is  met  by  the 
specific  system  characteristic  in  question.  This  is  in  fact  true  for  both 
cases.  However,  the  key  to  the  effective  use  of  this  information  is  in  the 
mission-derived  simulation  requirements.  For  example  .( refer  to  Figure  22), 
independent  motion  of  targets  within  the  visual  field  is  not  considered  a 
necessary  feature  of  the  low  altitude  visual  navigation  problem.  Even  though 
the  inability  of  the  open-loop  movie  system  to  provide  such  motion  is  a 
primary  disadvantage  of  the  system  per  se,  it  does  not  constitute  a disad- 
vantage in  achieving  the  simulation  objectives  of  this  mission.  Similarly, 
because  the  ability  to  make  small  localized  changes  in  the  stored  imagery 
is  not  required  in  this  case,  there  is  no  discrepancy  between  the  simulation 
requirements  and  system  capability  concerning  the  ease  of  change 
characteristic. 

c.  The  computer-based  imagery  system  is  certainly  the  most 
flexible  and  therefore  potentially  the  most  useful  of  all  the  simulation  tech- 
nologies.  However,  its  extreme  flexibility  is  both  its  greatest  strength  and 
its  greatest  weakness . As  mentioned  previously,  one  of  the  greatest  ad- 
vantages of  computer-generated  imagery  is  that  the  available  resolution 
elements  ("edges"  or  "lines11  depending  on  the  specific  technique  involved) 
can  be  differentially  apportioned  according  to  the  areas  of  interest  in 

the  visual  scene  . For  example  , in  an  air-to-air  combat  simulator  , most 
of  the  available  resolution:  elements  could  be  devoted  to  defining  the  image 
of  other  aircraft  in  the  visual  field  at  the  expense  of  ground  detail,  which 
is  only  needed  for  gross  orientation.  While  this  capability  is  a distinct 
advantage  of  the  computed  image  technique , implementing  it  presents 


NAVTRADEVCEN  69-C-0298-1 


a major  problem.  In  Figure  26,  tbis  is  directly  expressed  in  the  detail  re- 
quirement characteristic,  and  indirectly  reflected  in  the  high  software  cost 
factor.  It  will  also  be  seen  in  Figure  26  that,  of  the  system  characteristics 
which  were  previously  designated  as  critical  factors,  the  computer-genera- 
ted technique  is  deficient  in  only  one,  i.e.  , angle  of  vie w . However,  the 
highly  negative  detail  requirement  factor  should  be  regarded  as  a special 
disadvantage  of  this  technique  because  of  its  influence  on  the  values  of  the 
closely  related  scaling,  resolution,  color,  depth  and  cost  factors.  Obvi- 
ously, in  any  system  based  on  the  manipulation  of  a mathematically  defined 
visual  environment  each  element  of  that  environment  must  be  carefully  pre- 
selected and  incorporated  with  all  of  its  arbitrarily  assigned  parameters 
(such  as  color)  into  the  mathematical  model.  The  precise  definition  of  all 
the  visual  cues  necessary  to  meet  the  training  requirement  for  a given 
mission  is  often  such  a difficult,  expensive  and  time  -consuming  task  that 
it  constitutes  a major  disadvantage  of  the  approach  in  itself.  Figure  27 
illustrates  the  effect  of  individual  vs.  team  participation  in  the  analysis  of 
the  same  simulation  problem.  This  profile  is  based  on  the  combined  or 
team  judgments  of  two  individuals  (a  human  factors  specialist  and  a systems 
engineer)  in  determining  the  ability  of  the  CCTV-model  system  to  meet  the 
simulation  requirements  of  the  low  altitude  navigation  mission.  Actually, 
two  separate  profiles  are  depicted  in  Figure  27  . The  solid-line  profile 
represents  the  combined  judgments  of  both  individuals  , while  the  dashed 
line  portions  of  the  figure  show  where  the  profile  derived  solely  by  the 
human  factors  specialist  (Figure  26)  differs  from  the  combined  profile.  It 
should  be  noted  that  of  the  twenty  three  system  characteristics  evaluated, 
the  individual  and  the  team  judgments  were  identical  in  nineteen  cases. 

The  four  points  where  the  analyses  differ  illustrate  the  value  of  team  parti- 
cipation, in  that  these  differences  can  be  attributed  to  an  increased  precision 
in  the  knowledge  on  which  the  judgments  are  based.  For  example,  in  the 
team  context  it  was  pointed  out  that  considering  the  pature  of  low  altitude 
navigation,  an  unprogramed  simulation  capability  is  not  essential;  in  fact, 
a s emi  — unpr o grame d approach  is  adequate  to  meet  the  requirement.  Exam- 
ination of  the  appropriate  decision  matrix  of  Figure  22  illustrates  how  this 
change  or  refinement  of  the  simulation  requirement  affects  the  obtained 
matrix  score.  Since  the  system  characteristic  of  the  CCTV-model  approach 
does  not  change  (i.e.  , it  is  capable  of  providing  completely  unprogramed 
operation)  the  change  in  simulation  requirement  from  unprogramed  to  semi- 
programed  shifts  the  obtained  matrix  score  from  "0"  to  +1. 

The  analytic  technique  which  has  been  described  and  illus- 
trated here  was  developed  to  meet  a specific  and  long-standing  need,  that 
is  , the  need  for  a logical  and  systematic  method  of  organizing , displaying, 
and  evaluating  the  technical  information  which  is  already  available  con- 
cerning the  interaction  of  visual  simulation  requirements  and  techniques  . 

It  is,  of  course,  recognized  that  the  instrument  described  here  is  imperfect 
in  many  ways.  However,  the  basic  approach  employed  is  believed  to  be 
sound  and  logically  defensible  and  can  and  should  be  refined  by  practical  use 
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Figure  27.  Team  Judgments  on  the  Ability  of  the  CCTV  Model 
System  to  Meet  Simulation  Requirements. 
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Certainly  the  problem  to  which  this  technique  is  addressed  is  difficult  and 
persistent  and  will  not  be  overcome  without  effort, 

3,1.4  Research  Issues. 

3.  1.4.  1 Introduction.  Research  issues  in  the  field  of  visual  simulation 
may  be  considered  to  derive  from  two  major  considerations.  The  first 
consideration  is  that  no  technique  recreates  the  real  visual  world  in  its 
entirety  and  with  all  its  degrees  of  freedom.  A simulation  technique  is 
selective  of  some  part,  some  aspect,  or  some  abstraction  of  the  visual 
world;  the  simulation  is  less  than  the  reality.  Hence,  a range  of  questions 
centers  around  the  issues  of  what  is  lost  to  the  human  observer  in  one  tech- 
nique as  compared  to  another  technique;  what  are  the  characteristics  of  the 
image  here,  and  what  there;  and  so  on.  The  second  consideration  is  that 
the  simulation  is  being  made  not  for  its  own  sake  but  as  a means  for  training 
the  person  to  perform  a task  involving  the  perception  of  visual  imagery. 
Hence,  there  is  a range  of  questions  concerning  the  needful  fidelity  and 
validity  of  the  representation  and  whether  positive  transfer  of  training  is 
occurring.  In  most  cases  these  two  considerations  are  intimately  linked. 

The  simulation  technique  is  chosen  with  the  chief  intention  of  providing  the 
needed  fidelity  in  the  scene  for  the  tasks  to  be  trained. 

The  theme  of  the  necessary  incompleteness  of  the  simulation  and 
degree  of  training  effectiveness  underlies  each  of  the  following  research 
issues  and  topics.  They  have  been  chosen  on  the  basis  that  if  knowledge  or 
technique  could  be  improved  in  the  areas  specified,  more  or  less  immediate 
improvements  could  be  expected  in  the  training  effectiveness  of  the  visual 
components  of  simulators. 

3.  1.4.2  Image  Quality  Measurement.  Image  quality  may  be  defined  in  a 
variety  of  quantitative  and  qualitative  ways  (Soule,  1968;  Luxenberg  and  Barne 
1965;  Ozkaptan,  et  al,  1969).  No  one  of  these  ways  seems  to  characterize 
or  to  predict  completely  the  response  of  the  human  observer  to  the  presenta- 
tion, The  Modulation  Transfer  Function  (MTF)  is  an  index  of  how  well  image 
details,  in  the  original  are  reproduced  in  the  display.  The  technique  is  a 
variation  of  methods  employed  to  assess  and  predict  television  and  camera 
system  performance,  and  in  other  applications.  It  is,  by  its  nature  , con- 
fined to  assessing  grey  scale  and  contour  sharpness  aspects  of  the  display. 

It  takes  no  account  of  the  content  or  pictorial  complexity  of  the  display. 

There  have  been  various  attempts  to  generate  descriptive  indices  of  displays. 
Bennett,  et  al  (1962),  took  image  quality  in  terms  of  resolution  as  the  index, 
and  while  it  was  true  that  better  recognition  performance  was  associated  on 
the  average  with  better  resolution,  yet  there  was  a very  considerable  vari- 
ation among  different  targets  , Ornstein,  et  al  (1961),  developed  a com- 
plexity model  based  on  the  numerosity  of  contours,  and  Johnston  ( 1968) , 
demonstrated  that  it  has  power  to  predict  the  recognition  time  of  targets  by 
human  observers,  at  least  in  relatively  simple  displays.  Thus,  while  a 
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good  start  has  been  made  and  techniques  exist  with  some  predictive  capa- 
bility, it  is  highly  desirable  to  extend  this  work.  An  effective  index  of 
image  quality  would  permit  the  accurate  specification  of  simulator  per- 
formance and  permit  a more  rational  tradeoff  between  the  various  pictorial 
parameters  . 

3.  1.4.3  Vision/ Motion  Interaction  and  Cue  Parity.  Motion  of  the  visual 
scene  implies  that  one  is  moving.  Increasingly,  actual  motion  (acceleration) 
is  being  introduced  into  simulators  to  increase  realism,  reduce  cue  conflict, 
and  in  general  to  maintain  cue  parity  between  all  the  inputs  impinging  on 
the  trainee.  Many  studies,  reviewed  by  Rock  (1966),  suggest  that  vision  is 
the  dominant  sense,  and  that  vision  tends  to  control  the  unified  percept 
which  a person  arrives  at  concerning  a scene  and  the  events  occurring  to 
him.  However,  all  the  other  senses  contribute,  at  the  least,  to  the  percep- 
tion and,  when  a stimulus  affecting  another  sense  is  sufficiently  strong,  it 
may  become  the  controlling  input.  A review  of  the  literature  reveals  no 
studies  which  measure  the  degree  of  discrepancy  which  can  occur  and  not 
be  noticeable  to  the  observer.  The  various  falsifications  which  persons 
experience  in  a simulation  are  due  sometimes,  no  doubt,  to  this  lack  of 
parity  between  the  visual  and  motion  cues. 

Disparity  occurring  between  the  visual  and  motion  senses  has 
been  suggested  as  the  cause,  or  a contributing  cause,  to  motion  sickness 
experienced  in  a simulator.  Studies  are  needed  to  measure  thresholds  of 
intersensory  parity  under  various  conditions  of  task  and  degree  of  motion, 
and  to  assess  whether  different  amounts  of  parity  are  associated  with 
sickness  in  the  simulator. 

3.  1.4.4  Distortion  and  Cue  Conflict.  Visual  distortion  and  cue  conflict 
are  present  in  some  form  in  every  simulation.  Little  is  known  about  the 
relative  effects  of  one  form  of  distortion  compared  with  another.  Experi- 
ence indicates  that  simple  "lacks11  or  "absences"  in  a scene  are  not  dis- 
turbing . The  absence  of  color  is  acceptable;  its  presence  adds  to  the  scene. 
The  absence  of  independent  motion  is  acceptable;  its  presence  adds  to  the 
scene.  A narrow  screen  view  is  acceptable;  a wide-screen  provides  a 
more  compelling  illusion.  Distortion  and  cue  conflicts,  however,  seem  to 
be  a common  cause  of  user  unacceptability  . Optical  distortion  of  the  verti- 
cals and  horizontals,  poor  resolution,  flicker,  and  numerous  other  "arti- 
ficial11 visual  effects  have  been  blamed  for  causing  simulator  sickness 
(Puig,  1970). 

Ruling  out  simulator  sickness,  there  remain  problems  of  cue 
distortion  and  conflict  in  terms  of  judgment  of  depth  and  the  perception  of 
the  horizontal  plane.  Not  all  cue  distortions  , conflicts  , or  insufficiencies 
can  be  eliminated.  There  is  a choice.  Little  is  known  concerning  how  to 
effect  this  choice. 
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3.  1.4.5  Training  and  the  Visual  Parameters.  There  is  a lack  of  data 
relating  visual  parameters  to  training  effectiveness.  No  study  was  found, 
for  instance,  relating  the  presence  of  color  vs.  black  and  white  to  training 
effectiveness.  There  is  a need  for  a systematic  series  of  studies  relating 
viewing  distance,  variations  in  resolution,  brightness,  contrast,  field  of 
view,  color,  image  size,  etc.,  to  their  effectiveness  for  training  purposes 
A psychological  axiom  is  that  the  human  can  generalize  and  transfer  his 
experience.  But  in  the  simulator  context,  we  are  ignorant  still  of  whether 
we  are  overdesigning  or  underdesigning  the  training  devices. 
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3.2  PLATFORM  MOTION  SIMULATION. 

This  chapter  presents  concepts  and  data  on  motion  simulation  pertinent 
to  the  design  of  ground-based  training  devices.  The  emphasis  is  placed  on 
information  describing  acceleration  and  deceleration  force  magnitudes  (i.e.  , 
rate  changes)  since  movement,  in  itself,  provides  insufficient  cues  for  the 
perception  of  motion.  It  is  not  any  steady  rate  of  velocity,  but  the  change  of 
rate  of  motion  that  man  perceives.  Because  of  the  severe  restrictions  on  the 
physical  motion  that  can  be  incorporated  into  ground-based  training  devices, 
the  means  for  achieving  the  acceleration  cues  relyin  part  on  deceptions  and 
subterfuge,  since  most  devices  cannot  be  moved  efficiently  more  than  several 
feet  from  their  normal  or  static  position.  Thus,  the  timing  and  force  com- 
ponents must  occur  in  such  ways  as  to  encourage  the  trainee  to  react  f,as  ifM 
the  relevant  and  actual  aircraft  motions  are  taking  place. 

The  issues  and  data  of  interest  to  the  human  factors  specialist  in.  the 
design  of  training  devices  , are  concerned  with  short  duration  acceleration 
forces  that  are  meaningful  in  terms  of  perceptual  thresholds,  hence  relying 
on  onset  and  washout  rates  for  these  forces,  on  the  scaling  down  of  the  physi- 
cal forces,  and  on  augmenting  physical  motion  with  other  motion  cues  to 
achieve  the  desired  simulation.  Additional  issues  relate  to  the  complexities 
of  the  acceleration  force  combinations  required  in  order  that  the  trainee 
accept  the  results  as  realistic  since  these  forces  affect  his  proprioceptive, 
vestibular,  and  tactual  mechanisms  differentially. 

The  human  is  incapable  of  differentiating  components  of  force  to  which 
he  is  exposed.  It  is  only  the  common  resultant  of  the  effects  of  various 
forces  which  he  can  sense.  Since  the  force  of  gravity  has  been  the  predomi- 
nant experience  of  man,  the  perception  of  the  effects  of  gravity  plays  an  im- 
portant role  in  his  subjective  orientation  in  three-dimensional  space.  For 
example,  when  accelerative  forces  are  added  to  the  force  of  gravity  so  that 
the  direction  of  action  of  the  resultant  force  is  changed  with  respect  to  the 
body,  the  sensation  is  that  of  being  tilted  in  space  (Brown,  1961). 

Conversely,  a number  of  motion  issues  of  prime  concern  in  the  aero- 
space environment  are  of  minimal  importance  to  simulation  design  for  train- 
ing. These  deal  primarily  with  the  physiological  reactions  to  acceleration 
stresses,  with  human  performance  under  high  g (including  impact)  loadings 
and  under  sustained  accelerations,  and  with  the  effects  of  excessive  or  sus- 
tained vibrations  on  human  performance.  Also,  no  emphasis  is  placed  on 
physiological  concomitants  of  motion,  e.g.  , disorientation,  nausea,  coriolis 
effects,  etc.  Much  research  has  been  accomplished  in  military  organizations 
on  issues  relevant  to  the  perception  of  motion  by  man  and  on  the  effects  of 
accelerative  forces  on  human  performance  both  in  the  laboratory  and  in  the 
flight  simulator  contexts  . Representative  of  these  organizations  are  the 
National  Aeronautics  and  Space  Administration , particularly  NASA  Ames 
Research  Center  which  has  accomplished  considerable  research  on  motion 
employing  sophisticated  simulation;  the  Naval  Air  Development  Center 
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( Johns ville , Pennsylvania);  the  Naval  School  of  Aerospace  Medicine 
(Pensacola,  Florida);  and  the  USA.F  School  of  Aerospace  Medicine  (Brooks 
Air  Force  Base,  Texas).  Considerable  research  data  exist  in  the  open 
literature.  In  addition  to  individual  reports,  information  channels  enjoying 
wide  dissemination  include  such  sources  as  the  Journal  of  Aerospace 
Medicine  and  the  Human  Factors  Journal.  There  are  a sizeable  number  of 
compilations  and  summaries  of  research  and  application,  for  example,  com- 
pendiums  by  Roth  (1968),  Webb  (1964)  and  Brown  (1961);  books,  for  example, 
the  volume  edited  by  Burns,  Chambers  and  Hendler  (1963);  and  reports  pub- 
lished by  the  National  Academy  of  Science,  National  Research  Council,  for 
example.  Human  Acceleration  Studies  (NRC  Committee  on  BioAstronautic  s , 
1961). 


It  is  not  our  intent  to  review  this  voluminous  body  of  literature.  Our 
purpose  is  to  extract  from  this,  concepts,  data  and  operating  practices 
relevant  to  the  human  factors  design  of  training  devices.  Although  motion 
simulation  is  a design  option  for  a number  of  classes  of  air,  surface,  and 
subsurface  vehicles,  the  severest  challenge  for  human  factors  design  is  the 
simulation  of  motion  representing  an  air  vehicle.  While  the  discussion  in 
this  section  tacitly  assumes  a variety  of  vehicle  classes,  the  concern  cen- 
ters on  motion  simulation  for  aircraft  systems. 

The  ensuing  presentation  is  organized  as  follows: 

, An  elemental  review  of  the  motion  forces  pertinent  to  the 
simulated  environment  is  provided. 

. A review  of  the  literature  is  presented  for  the  purpose  of 

placing  into  perspective  the  utility  of  moving  base  simulation 
for  training  effectiveness  . 

. Motion  requirements  relative  to  training  device  design  are 

organized.  ' 

V Specifications  for  the  incorporation  of  motion  into  ground- 
based  training  devices  are  presented  to  the  extent  of  the 
data  available  . 

. Based  on  the  status  of  the  human  factors  technology,  re- 
search issues  are  identified  for  improving  knowledge  and 
technique  for  design. 


3.2  .1  Motion  Forces  in  the  Simulated  Environment.  Various  nomencla- 
ture  is  in  use  to  describe  the  acceleration  environment.  The  unit  G is  used 
to  describe  the  physiological  acceleration;  the  unit  _g  define s true  displace- 
ment acceleration.  The  physiological  acceleration  represents  the  total 
reactive  force  divided  by  the  body  mass  , and  hence  include  s both  displace - 
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acceleration  axes  represent  directions  of  the  reactive  displacements  of  organs 
and  tissues  with  respect  to  the  skeleton.  The  Z axis  is  down  the  spine,  with 
+GZ  (unit  vector)  designations  for  accelerations  causing  the  heart,  etc.  , to 
displace  footward  (caudally).  The  X axis  is  front  to  back,  with  +GX  desig- 
nations for  accelerations  causing  the  heart  to  be  displaced  back  toward  the 
spine  (dorsally).  The  X axis  is  right  to  left,  with  +Gy  designations  for 
accelerations  causing  the  heart  to  rotate  (roll)  to  the  left.  Angular  accel- 
erations which  cause  the  heart  to  rotate  (roll)  to  the  left  within  the  skeleton 
are  specified  by  the  +RX  unit  vector,  representing  radians/sec^  about  the 
X axis.  Angular  velocities  in  the  same  sense  are  specified  by  the  +RX  unit 
vector,  representing  radians /sec  about  the  X axis.  Similarly  +Ry  represents 
an  angular  acceleration  producing  a pitch  down  of  the  heart  within  the  skeleton 


and  +RZ  represents  yaw  right  of  the  heart  within  the  skeleton  (Roth,  1968). 
The  relationships  are  shown  in  Figure  28  . 
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The  nomenclature  for  describing  vehicle  displacement  accelera- 
tion defines  displacements  in  terms  of  three  vehicular  axes,  X,  Y and  Z. 
These  acceleration  axes  refer  to  the  center  of  gravity  of  the  vehicle,  where 
X is  the  longitudinal  axis;  Y is  the  lateral  axis;  and  Z is  the  vertical  axis. 
The  acceleration  unit  (g)  is  feet  per  second  per  second  (ft/ sec^)  regardless 
of  direction. 


Aircraft  motions  in  flight,  vis-a-vis  the  direction  of  acceleration, 
are  described  in  terms  of  rotational  and  translational  movements.  These 
can  be  analyzed  into  six  different  components  (i„e.,  degrees  of  motion 
freedom),  involving  movement  along  and  around  the  longitudinal  axis  , the 
side-to-side  axis,  and  the  up-down  axis  (Figure  29).  The  terminology  is 
as  follows: 


YAW 
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rotation  (angular  motion) 


roll  (bank  about  the  longitudinal  axis) 
pitch  (nose  up-down  about  the  lateral  axis) 
yaw  (heading  change  about  the  vertical  axis) 

translation  (linear  motion) 


surge  (fore  and  aft  along  the  longitudinal  axis) 
sway  ( side -to  - side  in  the  lateral  axis) 
heave  (seat  to  head  in  the  vertical  axis) 


At  any  given  time,  the  total  linear  acceleration  of  the  center  of  gravity  of 
the  vehicle  is  the  resultant  of  the  three  vectors  of  surge,  sway  and  heave. 
Angular  acceleration  components  about  any  given  linear  axis  can  be  ex- 
pressed in  radians  per  second  per  second  (1  rad/sec^  equals  57 . 3°/sec^), 
with  p,  g and  r indicating  the  directions  of  rotation. 

Rotations  Translations 

Roll  (bank)  Surge  (fore -aft) 


p (about  longitudinal  axis) 
Pitch  (nose  up/ down) 

q (about  lateral  axis) 

Yaw  (heading) 

r (about  vertical  axis) 


longitudinal  (4*  Gx) 
Sway  (side -side) 
lateral  (-f-Gy) 
Heave  (seat-head) 
vertical  (jf  Gz) 


It  is  reasonable  to  use  the  NASA  vehicle  displacement  acceleration 
terminology  in  specifying  human  acceleration  experiences  (however,  com- 
plete agreement  on  this  terminology  has  not  yet  been  reached,  see  for  ex- 
ample, Clark,  Hardy  andCrosbie,  1961).  Figure  30  provides  a description 
of  the  vehicle  acceleration  environment.  The  equations  of  motion  of  the 
aircraft  can  be  written  in  terms  of  these  six  vector  notations  . Table  18 
provides  a summary  of  acceleration  nomenclature  in  frequent  use  (Chambers 
1963). 


3.2.2  Utility  of  Motion  Simulation.  There  is  ample  anecdotal  informa- 
tion and  a growing  experimental  base  of  evidence  that  incorporating  dynamic 
motion  cues  into  simulators  used  forr|iaihing  produces  a change  in  trainee 
behavior  from  that  obtained  with  fixeM'i^^Sse  simulation.  The  values  of  pro- 
prioceptive, vestibular  and  tactile  cues  provide  desired  feedback  to  the 
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Nasa  Airplane  Axis  System 
(Vehicle  Displacement) 

LINEAR  ACCELERATION  MODES 


Description  of  Aircraft  Motion 
Acceleration  forward  (surge) 

Deceleration  (Accelerate  backward) 
Downward  Acceleration 
Upward  Acceleration  (heave) 

Straight  and  level  flight  at  constant  speed 
Acceleration  to  right  (sway) 

Acceleration  to  left 

a = ax  + ay  + °z 


Symbol  Unit 

+ax  ft/sec2  or  g 

— ax  ft/seca  ar  g 

+az  ft/sec2  or  g 

— az  ft/sec3  or  g 

°*  = ° 

+ay  ft/ sec2  or  g 

— ay  ft/sec2  or  g 


ANGULAR  ACCELERATION  MODES 

Angular  acceleration  about  X-axis  (roll  right)  p rad/sec9 

Angular  acceleration  about  Y-axis  (pitch  up)  q rad/sec9 

Angular  acceleration  about  Z-axis  (yaw  right)  f rad/sec2 


Figure  3 0.  Description  of  the  Vehicle  Acceleration  Environment 
(from  Clark,  Hardy  and  Crosbie,  1961 ) 
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TABLE  18.  SUMMARY  OF  ACCELERATION  NOMENCLATURES  FREQUENTLY 
USED. 
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(from  Chamber Sj  1963) 
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trainee  of  vehicle  status  that  cannot  be  obtained  from,  the  other  senses. 

The  acceleration  sensors  are  continuously  at  work  monitoring  events  so 
that  an  attention  channel  is  not  needed  to  detect  accelerations.  These 
sensors  are  also  sensitive  in  detecting  acceleration  changes.  Since  acceler 
ation  occurs  prior  to  any  velocity  change,  acceleration  cues  alert  the  pilot 
to  the  imminent  occurrence  of  a motion. 


A number  of  studies  have  examined  the  addition  of  motion  cues  in 
flight  simulators  and  have  compared  performance  with  and  without  the 
motion  capability.  A summary  of  pertinent  studies  is  presented  here  for 
the  purpose  of  placing  into  perspective  two  questions:  1)  does  moving  base 

simulation  facilitate  training,  and  2)  if  the  addition  of  motion  provides  train- 
ing advantage,  how  and  to  what  extent  is  this  advantage  manifested?  Since 
much  has  been  written  on  comparisons  of  fixed  vs.  moving  base  simulation, 
the  reader  is  encouraged  to  examine  collaterally  a number  of  reports  avail- 
able in  the  literature  that  review,  for  varying  purposes,  the  research  on 
motion  simulation.  Representative  of  these  are  the  reports  by  Graham 
(19&8)  , Peters  (1969),  Matheny  and  Wilker son  ( 1966)  , Cohen  (1970)  and 
Fuig  (1970).  A number  of  studies  have  examined  the  addition  of  motion 
cues  in  flight  simulators  and  compared  performance  with  and  without  motion . 
The  research  findings  suggest  that  motion  cues:  1)  serve  to  aleyt  the  pilot 

to  changing  states,  2)  encourage  the  buildup  of  correct  habit  patterns  in 
training  , 3)  increase  the  realism  of  the  simulation  thereby  enhancing  trainee 
confidence  in  the  devic e 4)  improve  pilot  responses  , 5)  affect  aircraft  con- 
trol tasks,  6)  are  differentially  useful  as  a function  of  the  skill  level  of  the 
pilot,  and  7)  vary  in  relevance  (parameters  and  values)  as  a function  of  task 
or  mission  requirements.  The  essence  of  these  findings  , based  on  repre- 
sentative studies,  are  summarized  briefly  below. 


Motion  enhances  the  pilot Ts  ability  to  control  the  aircraft.  In  simu- 
lated landings  of  an  aircraft  with  roll  disturbances,  an  average  of  about  8 
seconds  was  required  to  restabilize  the  simulator  with  a moving  base;  about 
twice  the  time  was  required  with  a fixed  base  simulator  (Young,  1967).  The 
continuous  motion  cueing  makes  the  pilot  more  aware  of  the  requirements 
for  , and  the  consequences  of,  control  inputs  and  enables  him  to  perform 
maneuvers  more  precisely  and  quickly  (Perry  and  Naish , 19  64)  . A study 

involving  tracking  in  pitch  during  terrain  contour  flying  in  a simulator  with 
and  without  motion,  found  that  motion  cues  facilitated  precise  tracking  per- 
formance (Be sc o , 1 9 61)  . 


In  an  early  study,  a 4—  degree  -of-freedom  motion  condition  was 
compared  with  a no -motion  condition  in  a Link  ME  - 1 ( T - 38 ) jet  trainer. 
Six  experienced  pilots  performed  a night  approach  and  landing  task  with  a 
c r os s wind  disturbance  1 The  visual  scene  was  simulated  by  a computer- 
generated  display  of  all  required  light  patterns  on  a CRT  monitor  mounted 
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outside  the  cockpit  (Wendt,  et  al,  1961).  * Without  motion  there  was  a 
(relative)  tendency  to  undershoot  the  runway;  however,  lateral  deviations 
were  kept  small,  With  motion,  the  simulated  aircraft  touched  down  signi- 
ficantly later;  at  the  same  time,  the  lateral  correction  was  less  adequate 
according  to  the  automatic  plotting  device  used  to  monitor  objective  per- 
formance. Evidently,  motion  alerted  the  pilot  to  certain  more  critical 
problems  and  caused  a different  allocation  of  his  information- processing 
capabilities  . 


■t 


i; 


In  situations  where  pilot- induced  oscillations  (PIO)  are  of  concern 
(e.g.  , high  performance  aircraft  with  fully  powered  control  systems),  mov- 
ing base  simulation  is  mandatory  since  PIO  is  difficult  to  elicit  in  fixed  base 
simulation  (Rathert,  Greer  and  Douvillier,  1959). 

Simulated  motion  in  three  degrees  of  freedom  (pitch,  roll  and  ver- 
tical translation  on  the  Grumman  Multipurpose  Simulator)  provided  signifi- 
cantly enhanced  transfer  to  the  criterion  task  in  the  simulator  as  compared 
to  the  performance  of  people  trained  without  motion.  The  criterion  task 
was  described  as  a simulated  high-speed  low-altitude  mission  through  clear 
air  turbulence  (Buckhout,  et  al,  1963). 

Motion  cues  enabled  pilots  to  perform  a precise  formation  flight 
task  over  a range  of  longitudinal  stability  conditions  better  under  two  con- 
ditions of  stick  force  gradients  (one  gradient  being  zero)  . The  value  of 
motion  cues  in  enhancing  performance  compared  to  no  motion  was  particu- 
larly dramatic  when  stick  force  gradients  were  zero  (Brown,  Johnson  and  ,r;i; 
Mungall,  i960)  . 

A study  of  a simulated  carrier  approach  and  landing  maneuver 
using  the  Grumman  Multipur pos e Simulator  investigated  performance  under 
static  (no  cockpit  motion)  and  kinetic  (cockpit  motion)  conditions  . Kinetic 
cueing  during  training  significantly  improved  transfer  task  performance , 
where  statically  and  kinetically  trained  groups  transferred  to  the  motion 
condition  in  the  simulator  (i.e.  , training  and  transfer  tasks  were  performed 
in  the  simulator)  . A conclusion  was  that  motion  functions  as  a ’’general 
alerter11  and  hence  serves  to  alert  the  pilot  to  a changing  state  that  prompts 
increased  attention  to  visual  cues;  in  certain  cases  , motion  may  provide 
specifically  useful  information  that  the  pilot  uses  directly  for  controlling 
his  vehicle  (Ruocco,  Vitale  and  Benfari , 1965). 


^ Wendt ,■  H.  W.  , Stark,  E . A . / Simon,  G.  B.  , and  Cohen,  E . The  value  of 
cockpit  motion  in  flight  simulation:  an  experimental  approach.  Paper 

read  at  the  69th  Annual  Meetings  of  the  American  Psychological  Associ- 
ation.  Session  or  Engineering  Psychology  I.  New  York:  September , 19  6 1. 
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Differences  in  pilot  performance  in  simulators  with  and  without 
motion  cues  are  quite  pronounced  in  flight  under  heavy  turbulence  conditions 
or  for  aircraft  with  high  natural  frequencies  and  low  damping.  Motion  aids  ] 

in  the  building  of  a response  pattern  which  transfers  positively  to  the  air-  j 

craft.  Simulator  training  under  severe  turbulence  conditions  is  highly  j 

desirable  because  of  the  demanding  and  hazardous  nature  of  the  flight  and  j 

because  pilots  do  not  have  many  opportunities  for  in-flight  training  under 
turbulence  conditions,  hence,  are  ill-prepared  to  cope  with  the  situation. 

The  more  degrees  of  freedom  of  motion  installed  in  the  device  the  better 
the  training  capability  since  turbulence  affects  an  aircraft  in  all  motion 
dimensions. 


When  a sudden  disturbance  is  applied  in  the  simulator,  the  feed- 
back loop  involves  the  vestibular  system  which  immediately  alerts  the  pilot 
to  the  disturbance,  although  not  to  its  exact  nature.  His  response  is  to  scan 
the  instruments  to  detect  any  change.  Since  there  will  be  a delay  after  the 
acceleration  before  the  instruments  show  the  effects  of  the  disturbance,  the 
pilot  is  primed  to  attend  to  any  discernable  change  and  apply  the  indicated 
control  corrections  . He  is  thus  able  to  predict  what  is  going  to  happeM  t*£- 
the  simulator  by  means  of  these  feedback  loops  and  hence  build  up  habit 
patterns  similar  to  those  required  in  the  aircraft.  In  simulators  without 
motion,  a pilot  is  deprived  of  fast  acceleration  feedback  and  so  ouilds  up 
simulator  habit  patterns  which  are  not  equivalent  to  those  demanded  in  the 
air  (Borlace  , 1967) 


The  importance  of  motion  cues  appears  to  be  a function  of  the  ex- 
perience level  of  the  pilot,  but  whether  motion  in  the  simulator  is  more 
important  for  initial  skill  acquisition  or  for  training  pilots  of  some  experi- 
ence will  depend  upon  the  task  among  other  things.  Some  data  suggest  that 
motion  cues  are,  at  least,  more  used  and  relied  on  as  experience  increases  . 
Graham  ( 1968)  believes  that  motion  serves  as  secondary  cues,  normally 
used,  and  improves  the  performance  of  experienced  pilots  since  they  under- 
stand the  relationship  of  the  motion  to  the  task.  Conversely,  motion  degrades 
the  performance  of  inexperienced  trainees  since  the  motion  is  perceived  as. 
an  unrelated  distraction  which  hinders  the  control  capability.  The  omission 
of  motion  in  the  simulator  forces  the  pilot  to  alter  his  behavior  from  that 
found  in  the  air  in  that  he  attends  to  the  primary  signal  more  closely , makes 
larger  and  less  frequent  control  movements  , and  accepts  greater  error  . The 
experienced  pilot  relies  on  associations  between  motion  cues  and  responses 
learned,  in  actual  flight  which  alerts  him  to  attend  to  certain  instruments 
when  certain  motions  are  felt.  : / . 


Motion  cues  are  more  closely  coupled  to  the  pilot  than  a r[e  visual 
cues  . The  visual  sense  is  not  as  precise  in  sensing  movement  as  the  motion 
senses  . Motions  sensed  kinesthetic  ally  act  quickly  and  directly  on  the  pilot. 
Matheny,  Dougherty  and  Willis  (1963)  state  that  the  proprioceptive  senses 
lead  the  visual  sense  in  perceiving  the  development  of  certain  conditions  of 


ERIC 


.237 

- 225  - 


sags 


NAV TRADE VCEN  69-C-0298-1 


flight.  Their  data,  applicable  to  a light,  single  rotor  training  helicopter, 
indicated  that  when  the  acceleration  reached  about  20°/sec^  motion  cues 
preceded  the  visual  cues  by  a significant  amount.  When  the  angular  accel- 
eration reached  approximately  25°/sec2,  the  motion  cues  preceded  the 
visual  cues  by  .4  to  .5  seconds,  or  the  order  of  magnitude  of  one  reaction 
time.  Thus,  motion  cues  alert  and  direct  the  attention  of  the  pilot  to 
changes  in  the  aircraft  enabling  him  to  interpret  displays  and  take  action 
for  maintaining  aircraft  control. 

Studies  at  the  Man- Vehicle  Control  Laboratory  of  the  Massachusetts 
Institute  of  Technology  (reported  by  Graham,  1968)  indicate  the  following. 


• In  controlling  a stable  closed-looped  system,  visual 
information  is  most  important.  As  stability  is  de- 
creased, rapid  detection  of  changes  in  orientation 
becomes  more  important  than  accuracy  of  alignment, 
and  motion  cues  become  most  important. 

• Motion  cues  assist  the  pilot  in  Mtaking  over"  following 

a failure  in  an  automatic  blind  landing  system.  A 
nearly  100  percent  deterioration  (i . e . , increase  in 
recovery  interval)  occurred  when  motion  cues  were 
not  available  in  simulation.  Re  stabilization  of  the 
aircraft  after  the  introduction  of  roll  disturbances 
required  an  average  of  8 seconds  with  moving  base 
simulation  and  approximately  12-16  seconds  with 
fixed  base  simulation.  1 f 


* Helicopter  pilots  rely  on  motion  cues  for  attitude 
stabilization . Use  of  motion  cues  appears  directly 
proportional  to  pilot  experience , with  more  skillful 
pilots  benefiting  more  from  motion. 

Evidence  from  a series  of  studies  for  the  Army-Navy  Instrumenta- 
tion Program  on  helicopter  simulation  (Feddersen,  1962)  indicated  that 
skilled  pilots  trained  on  a moving  base  simulator  (pitch,;  roll,  heave  and 
yaw  cues)  achieved  a greater  rate  of  acquisition  of  proficiency  and  a higher 
level  of  performance  (asymptote)  compared  to  fixed  base  training  conditions  . 
The  performance  of  the  group  initially  trained  on  motion  deteriorated  when 
transferred  to  the  no  - motion  condition;  the  performance  of  the  group  initially 
trained  under  no -motion  improved  significantly  when  transferred  to  the  motion 
condition.  Similar  results  (i . e . , attaining  greater  proficiency  more  quickly 
under  conditions  of  motion)  were  demonstrated  by  naive  subjects.  It  was 
concluded  that  a training  device  incorporating  the  above  4 degrees  of  motion 
freedom  would  enhance  the  rate  of  proficiency  attainment  and  increase  the 
amount  of  transfer  obtained.  ; . m:  :/ 


O 
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Control  performance  is  in  any  event  different  and,  often,  more 
precise  with  integrated  visual  and  motion  cues  than  with  visual  cues  alone  . 
The  pilot  receives  stimuli  both  visxxally  and  kinesthetic  ally,  responds  with 
an  initial  control  action  and  then  adjusts  his  response  based  on  the  feedback 
signals  provided  through  the  multiple  sensing  channels  . In  a study  which 
simulated  aircraft  control  in  a side  gust  employing  out-of- the -cockpit  visual 
scenes  with  and  without  motion,  the  effect  of  the  motion  increased  pilot  re- 
action and  control  accuracy.  Time  lags  for  the  pilot  response  ranged  from 
17  seconds  with  no  motion  cues  to  about  1/2  second  when  motion  cues  were 
available  (Perry  and  Naish,  1964).  Bristow  (n.  d.)  concurs  that  motion 
cueing  increases  the  pilot*s  response  frequency  and  significantly  improves 
his  control  accuracy  at  very  low  frequencies.  The  succession  of  motion 
cues  makes  him  more  aware  of  control  input  requirements'  and  of  the  conse- 
quences of  his  responses.  In  Feddersen's  study  (1962),  the  pilotTs  control 
movements  were  high  frequency  and  low  amplitude  when,  motion  cues  (yaw, 
heave,  pitch  and  roll)  were  available  , and  low  frequency,  high  amplitude  in 
the  fixed  base  simulator.  The  effects  of  motion  cues  quickened  the  percep- 
tion of  and  response  to  error.  Motion  cues  enabled  the  pilot  to  sense  the 
rate  of  onset  of  a deviation,  thus  error  was  perceived  pr oprioceptively  in 
advance  of  any  visual  cues  . In  a study  conducted  at  Douglas  Aircraft 
Company,  availability  of  motion  cues  (in  a Link  DC-8  simulator)  enabled 
experienced  pilots  to  detect  simulated  hydraulic  brake  failures  quickly  dur- 
ing the  landing  so  that  emergency  brakes  were  operated  within  1.5  to  3 
seconds  . With  the  motion  system  of  the  simulator  turned  off,  it  took  the 
pilots  more  than  7 seconds  to  operate  the  braking  system  (cited  by  Cohen, 
1970). 


3.2.2.  1 The  Consensus  . The  considerable  evidence  indicates  that  train- 
ing capability  is  influenced  substantially  by  the  presence  of  physical  motion 
in  the  simulator.  The  consensus  is  that  motion  cues:  provide  significant 

assistance  to  the  pilot  in  aircraft  control;  alert  the  pilot  to  changes  develop- 
ing in  the  state  of  the  vehicle;  and  provide  an  extension  of  opportunity  to  the 
trainee  to  learn  by  exposing  him  to  additional  task  demands  . There  is  also 
some  evidence  that  motion  cues  contribute  to  the  development  of  habit 
patterns  analogous  to  those  required  in  actual  flight. 


Evidence  supporting  the  decision  to  incorporate  motion  into  the 
ground-based  trainer  is  yet  incomplete  in  terms  of  actual  transfer  benefits  . 
The  majority  of  the  findings  do  not  yield  clear  expressions  of  the  extent  of 
transfer  of  training.  It  is  not  enough  to  say  that  training  value  is  enhanced. 
A practical  issue  is  the  saving  in  aircraft  time  resulting  from  the  addition 
of  motion  cues  in  the  simulator.  Matheny  and  Wilke r son  (1966)  in  review- 
ing Fedder sen *s  ( 19 62)  data  indicate  a slight  advantage  for  the  moving  base 
simulator . Approximately  six  hours  in  a fixed  base  simulator  plus  twenty 
minutes  in  the  helicopter  will  produce  proficiency  equivalent  to  4-1/2  hours 
in  a moving  base  simulator  with  10  minutes  in  the  helicopter.  Howeve  r , 
the  study  does  not  accurately  test  the  extent  of  the  effects  of  motion  upon 
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transfer  of  training,  since  the  two  groups  did  not  receive  equal  amounts 
of  time  in  the  trainer.  The  suggestion  is  that  for  beginning  helicopcer 
trainees,  a proper  visual  display  and  thorough  training  will  yield  a high 
degree  of  transfer  without  motion.  Feddersen  (1961)  described  a study  of 
helicopter  training  in  which  an  experimental  group  was  trained  in  hovering 
on  a six  degree  of  freedom  simulator  and  a control  group  was  trained  under 
static  conditions.  Upon  reaching  an  asymptote  in  training,  each  trainee  was 
given  six  2 - minute  hovering  trials  in  the  helicopter.  The  motion  cue-trained 
group  performed  better  initially  than  did  the  no -motion  trained  group  but 
the  differences  disappeared  by  the  end  of  the  six-trial  flying  session.  Based 
on  this,  it  appears  that  a no-motion  simulator  trained  group  can  quickly 
learn  to  use  the  motion  cues  in  the  aircraft  ^assuming  relevancy  with  visual 
cues  and  equivalence  in  response  dynamics  between  simulator  and  aircraft). 

It  appears,  however,  that  motion  simulation  is  desirable  in  pro- 
viding cues  and  event  disruption  or  degradation  for  training  a variety  of 
flight  tasks. 

3.2  .3  Simulator  Motion  Requirements  . Working  from  the  evidence  that 
the  addition  of  physical  motion  enhance  3 the  training  capability  of  simulators  , 
the  issue  for  human  factors  design  is  to  define  the  nature  of  the  motion  cues 
to  represent.  The  concern  is  for  providing  the  needed  motion  cues  rather 
than  completely  duplicating  the  spectrum  of  aircraft  motions  . To  achieve 
motion  cues  that  are  perceptually  undistinguishable  from  those  experienced 
in  flight  vehicles  is  a particularly  taxing  requirement  because  of  limitations 
in  the  physical  displacement  of  the  simulator,  less  than  continuously  avail- 
able power  and  the  structural  stiffness  of  the  device.  Further,  the  require- 
ment is  predicated  upon  how  man  combines  visual,  somesthetic  and  vestibu- 
lar sensations  into  a perception  of  motion  and  sufficient  data  on  this 
integration  are  yet  unavailable  to  provide  a body  of  information  than  can  be 
used  effectively  in  human  factors  design.  Basically,  for  any  motion  system, 
the  specifications  include  the  following:  the  degrees  of  freedom  required, 

and  for  each  motion  axis  , the  displacement  (linear  , angular)  , and  maximum 
acceleration  required.  The  parameters  of  displacement,  acceleration,  and 
velocity  are  interrelated,  for  example,  a maximum  velocity  attainable  is 
limited  by  a given  displacement  and  maximum  acceleration. 

In  the  following  paragraphs,  we  will  discuss  the  relationship  of 
motion  cues  to  task  requirements,  examine  the  relevant  priority  of  the 
motion  axes  to  be  simulated,  and  describe  the  techniques  for  achieving 
motion  cues  in  simulators  . 

3.2 . 3.1  Motion  Cues  and  Task  Requirements.  Although  the  addition  of 

physical  motion  in  the  simulator  induces  changes  in  trainee  behavior  , these 
changes  may  not  have  an  appreciable  effect  on  task  performance  or  they  may 
have  a substantial  effect  on  transfer:  Assuming  that  motion  cues  are  desir- 

able in  the  training  device , the  question  is  , when  are  motion  cues  most 
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efficiently  employed  and  what  are  the  parameters  axid  combinations?  Motion 
cues  in  the  simulator  are  of  value  directly  as  a function  of  the  type  of  task 
presented  to  the  pilot.  The  performance  of  NASA  test  pilots  over  a wide 
range  of  steady  state  and  oscillatory  conditions  under  all  six  degrees'  of 
motion  freedom  indicates  that  motion,  cues  are  mandatory  when  they: 
l)contribute  to  improved  control  of  the  vehicle,  i.e.  , help  the  pilot  by 
supplying  a necessary  lead  or  anticipation  cue,  as  in  coping  with  a lightly 
damped  or  unstable  vehicle,  and  2)  interfere  with  satisfactory  performance, 
i.  e.  , hinder  the  pilot  in  making  a desired  control  motion,  as  in  using  a 
powerful  or  a sensitive  control  system.  (Rathert,  Creer  and  Douvillier, 
1959;  Rathert,  Creer  and  Sadoff,  1961). 

3 . 2 . 3 . 1 . 1 Tasks  Benefiting  from  Motion  Inputs  . Motion  cues  appear 

to  be  mandatory  (as  opposed  to  ’’nice  to  have”)  for  increasing  the  training 
capability  in  a number  of  defined  tasks,  In  these  situations,  the  motion  cues 
strongly  influence  the  trainee’s  performance;  in  some  situations  they  affect 
performance  adversely  in  that  greater  correspondence  with  the  operational 
environment  is  achieved  resulting  in  increased  task  difficulty.  In  the 
following  listing  of  tasks  and  task  conditions,  motion  cues  are  regarded  as 
mandatory  for  effective  training: 

* all  tasks  involving  higher  g forces  and  frequent 
maneuvering  . 

* low-altitude  , high-speed  flight- -acc ele rations  in 
terrain  following  maneuvers  increase  the  relevance 
of  the  task  and  its  difficulty  level. 

* pursuit  tracking  per  for  mane  e - -close -coupled 
tracking  with  motion  correlates  more  closely 
with  actual  flight  under  turbulent  air  conditions 
(particularly  clear  air  turbulence  penetration). 

. low-altitude  , high-speed  flight- -under  heavy  tur- 
bulence , aircraft  instruments  as  well  as  man’s 
motion  sensors  (particularly  in  the  absence  of  a 
visual  reference)  provide  misleading  information. 

For  example,  in  heavy  gusts,  loss  of  control  may 
occur  in  pitch  due  to  one  or  more  misleading  cock- 
pit indications  which  call  for  the  wrong  control 
input.  / 

. stability  augmentation  system  (SAS)  failure  - -this 
is  a low  probability  event  where  motion  (which 
degrades  the  pilot’s  ability  to  cope  with  the  failure) 
will  enable  the  pilot  to  make  the  proper  responses  ^ 

in  correcting  the  situation.  j ^ 
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• control  of  highly  sensitive  or  marginally  stable  flight 
vehicles -- motion  cues  facilitate  the  control  of  these 
types  of  systems  up  to  the  point  of  instability,  then 
motion  interferes  with  pilot  control  and  may  cause 
loss  of  control. 


• Non-normal  landing  approaches  (such  as  carrier 
landings)  and  including  abnormal  conditions  (such 
as  power  losses,  loss  of  SAS,  turbulence) 

• V/STOL  operations,  particularly  in  takeoff  and 
landing  modes  with  a marginally  stable  vehicle. 

• Pilot  induced  oscillations — motion  cues  are 
mandatory  in  order  to  provide  training  in  PIO 
correlated  with  in-flight  occurrences. 

3.2. 3.2  Priority  of  Motion  Axes  for  Training.  It  is  clear  that  motion 
requirements  vary  as  a function  of  task  requirements;  one  type  of  simulator 
is  not  able  to  represent  realistically  all  the  motions  of  the  varieties  of  air- 
craft and  missions  flown.  The  question  is,  what  components  of  motion  are 
most  needed,  vis-a-vis  the  defined  training  requirements. 


The  consensuses  that  motion  in  the  rotational  axes  is  more 
useful  to  the  pilot  than  motion  in  the  tra.nslational  axes.  Graham  (1968), 
interested  in  large  jet  aircraft,  identifies  pitch,  roll  and  heave  (vertical 
translation)  as  the  most  important  simulator  axos,  followed  by  sway  (lateral 
translation),  yaw  and  surge.  The  next-to-last-place  ranking  given  yaw  is 
due  to  the  cockpit  being  displaced  some  distance  from  the  center  of  gravity, 
thus  yaw  is  sensed  as  a sway  condition.  Where  the  cockpit  is  located  close 
to  the  center  of  gravity,  yaw  is  sensed  and  takes  precedence  over  sway. 
When  the  man  is  near  the  yaw  pivot  point,  the  rotational  element  of  yaw  is 
an  important  motion  axis.  Surge  (longitudinal  acceleration)  is  an  important 
fidelity  consideration  but  difficult  to  simulate.  As  a long- term  low-level 
acceleration  it  is  not  critical,  but  short-term  surge  ( e .g.,  buffeting)  appear 
mandatory  for  turbulence  simulation  and  stall  representation.  Graham 
(1968)  states,  a moving  base  simulator  for  research  purposes  , n should 
have  at  least  three  degrees  of  freedom:,  roll , pitch  and  heave  . A fourth 
degree  of  freedom,  sway,  permits  the  simulation  of  many  additional  flight 
tasks,  the  most  noteworthy  being  the  loss  of  thrust  in  wing -mounted  engines 
and  V/STOL  approach  and  hover  ...  . the  necessity  for  the  remaining  two: 
yaw  and  surge  currently  appears  doubtful . 11  (p.  24).  ; 


The  inclusion  of  yaw  and  surge  is  supportable  for  a research 
simulator  representing  a wide  range  of  aircraft  types  . In  the  simulation  of 
small;  aircraft,  yaw  is  more  important  than;  sway ® Also  , once  four  degrees 
of  freedom  are  achieved,  the  addition  of  yaw  and  surge  is  more  easily 
^ "'stalled ^ v v'; 
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For  helicopter  simulation,  pitch,  roll  and  heave  motions  are 
desirable.  Matheny  and  Wilkerson  (1966)  state: 


"When  one  considers  response  to  control  inputs  within  the  heli- 
copter he  finds  that  pitch  and  roll  angular  motions  must  precede  the  fore  and 
aft  and  lateral  translation  motions  . Thus  , control  over  these  angular 
motions  is  necessary  for  initiation  or  termination  of  translational  motions. 
Further,  angular  accelerations  about  the  center  of  gravity  in  general  have 
greater  magnitude  and  rates  of  onset  of  acceleration  than  do  accelerations 
and  rates  of  onset  of  the  total  mass  of  the  helicopter  in  translation.  Ex- 
ternal forcing  functions  such  as  wind  gusts  are  also  analyzed  to  be  more 
disturbing  to  the  pitching,  rolling,  yawing  and  vertical  motions  of  the  heli- 
copter than  to  the  fore  and  aft  or  lateral  translations.  If  sustained,  the 
external  forcing  function  of  a steady  wind  may  bring  the  helicopter  to  a 
considerable  rate  of  motion.  However,  the  acceleration  and  rate  of  onset 
of  this  acceleration  is  felt  to  be  generally  so  low  that  its  simulation  in  the 
trainer  is  not  necessary. 

The  fact  that  fore  and  aft  and  lateral  translations  are  controlled 
through  the  pitch  and  roll  dimension,  and  that  the  externally  produced  trans- 
lational motions  in  fore  and  aft  and  lateral  may  generally  be  such  that  the 
visual  cues  will  lead  the  proprioceptive,  argues  for  the  conclusion  that  the 
translational  motions  need  not  be  simulated. 

With  respect  to  translational  motion  along  the  vertical  axis,  we 
find  that  this  motion  is  controlled  quite  directly  by  movements  of  collective 
or  throttle  and  its  responses  to  these  control  inputs,  as  well  as  to  external 
forcing  functions  , can  be  quite  rapid.  At  the  same  time  the  visual  percep- 
tion of  vertical  motion  and  vertical  acceleration,  in  particular  , may  be 
quite  difficult.  A second  consideration  argues  for  the  inclusion  of  motion 
along  the  vertical  axis  in  the  trainer  . In  a basic  trainer  equipped  with  a 
visual  attachment  for  presenting  contact  visual  cues  , a situation  exists  in 
which  these  cues  become  ambiguous  and  difficult  for  the  pilot  to  differentiate. 
The  visual  differentiation  of  a change  in  altitude  from  a change  in  pitch,  or 
from  a fore  and  aft  translation,  is  difficult  for  the  student  pilot  and  may 
even  be  troublesome  for  the  experienced  pilot.  In  the  visual  scene  a de- 
crease in  altitude  will  cause  an  object  immediately  in  front  of  the  helicopter 
to  appear  to  move  away  from  it.  An  aft  translation  or  pitch  down  will  bring 
about  an  analogous  movement.  The  question  of  concern  is  whether  the 
addition  of  vertical  motion  to  the  trainer  will  assist  in  this  discrimination, 
and  our  deduction  is  that  such  proprioceptive  cues  would  be  desirable  in  the 
trainer.  ' -V  :'vV  ;v.  .y-- *:■ 'yy;'\ V-yy' ' 


With  respect  to  the  rotational  motions  in  the  trainer,  we  have 
implied  earlier  and  wish  to  make  explicit  that  the  pitch  and  roll  motions  are 
necessary  to  be  simulated  in  any  motion  platform.  Recommendations  with. 
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respect  to  yaw  motions  are  more  difficult  since,  at  present,  they  are 
generally  more  costly  to  incorporate  into  the  system.  On  the  one  hand, 
the  high  degree  of  cross  coupling  between  yaw  and  other  dimensions  of 
motion  of  the  system  and  its  consequent  sensitivity  to  control  inputs  in 
other  dimensions  argues  for  its  inclusion.  On  the  other  hand,  the  visual 
cues  for  changes  in  yaw,  allowing  as  they  do  for  a high  sensitivity  to  dis- 
turbances in  this  dimension,  would  argue  that  the  redundant  motion  cue  is 
not  necessary. 

The  question  of  simulating  the  motion  about  the  vertical  axis 
can  be  simplified  by  assuming  the  pivot  point  of  the  observer  will  be  about 
an  axis  located  through  the  planar  portion  of  the  body.  With  this  orientation 
of  the  pivot  point  the  only  cues  sensed  proprioceptively  will  be  those  due  to 
rotation  (within  reasonable  rates  of  displacement).  If  the  pivot  point  is 
shifted  aft  of  this  location,  the  operator  may  be  able  to  sense  both  rotational 
and  translational  motion.  . . .(The)  evidence  indicates  the  visual  sense  may 
be  more  sensitive  to  rotational  errors  than  to  the  proprioceptive  senses. 
Assuming  this  to  be  the  case,  motion  cues  for  yaw  errors  appear  to  be  of 
limited  value.  . .until  yaw  motion  simulation  costs  fall  we  must  conclude  that 
yaw  motion  in  the  light  helicopter  trainer  is  not  an  economical  investment.  " 
(pp.  27-29). 


An  analysis  of  motion  cue  priority  for  two  types  of  helicopters 
(XJH-1D  and  CH47A)  has  yielded  the  results  shown  in  Table  19  (Clausen, 
et  al,  1968)  . 


A position  held  by  the  Naval  Training  Device  Center  (Fuig,  1970) 
is  that  "pitch,  roll  and  lateral  translations  are  considered  the  primary 
motions  which  should  be  incorporated  into  a motion  system.  Yaw  motion, 
longitudinal  surge  and  vertical  heave  should  be  included,  cost  permitting, 
but  are  generally  less  important.  The  relative  importance  of  these  motion 
cues  will,  of  course,  be  dependent  upon  the  aircraft  characteristics  being 
simulated.  For  instance,  vertical  heave  will  be  extremely  important  in  a 
V/STOL  or  helicopter  simulator,  whereas  the  lateral  translations  will  not. 
In  a carrier  aircraft  catapult  simulator,  longitudinal  surge  would  assume 
a role  of  major  importance;  training  for  corrective  action  for  engine  failure 
would  require  yaw  in  the  motion  system"  (p.  44). 


Huddleston  (1966)  believes  that  pitch  and  roll  accelerations  and, 
to  a lesser  extent,  heave  acceleration  are  mandatory  motions  over  a range 
of  applications.  He  suggests  (as  hypotheses  based  on  the  research  litera- 
ture) the  following: 


a . ’’Yaw,  sway,  and  surge  accelerations  would  be  progressively 
more  difficult  to  justify,  in  terms  of  cost,  for  flight  training  associated 
with  conventional  lift  aircraft. 
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TABLE  19.  MOTION  CUE  SEQUENCE  OF  PRIORITY. 


C ontrol 

Flight 

Priority 

Motion 

Tasks 

Regime 

UH-1D 

CH-47A 

Pitch./  Roll 

Attitude  Control, 
Translation 

All  Flight 
Regimes 

1 

1 

Heave 

Collective 

Altitude 

Control 

3 

2 

Y aw/ 
Late  ral 

Collective,  Power 
Directional  Control 
Coordination 

Turns , 
Power 
Changes, 
Malfunctions 

(Yaw) 

2 

(Lateral) 

3 

Vibration 

Airspeed  Control, 
Stall  Cueing, 
Instrume  nt 
Slurring 

High  Speed 
Flying, 
Power 
Settling 

4 

4 

(from  Clausen,  et  al,  1968) 


b.  Yaw,  sway,  and  surge  accelerations  would  be  progressively 
less  frequently  useful  in  general-purpose  research  simulation  associated 
with  conventional  lift  aircraft. 

c.  Yaw  and  sway  accelerations  would  be  most  frequently  useful 
in  the  simulation  of  V/STOL  and  engine  failure  cases  . 

d.  The  more  gentle  flight  modes  (takeoff,  landing,  straight  and 
level  flight)  can  be  more  thoroughly  treated  in  static  simulators  than  the 
more  vigorous,  maneuvering  modes  (target  chasing,  terrain  xollowing , 

investigating  boundaries  of  pilot  acceptability). 

e.  Attitude  control  tasks  are  those  which  most  demand  that 
the  simulation  include  cockpit  dynamics,  and  are  those  which  can  most 
easily  yield  subversive  results  from  static  simulator  work. 

f.  The  reduction  of  any  delays  in  the  onset  timing  of  cockpit 
accelerations  in  two  or  three  axes  can  be  more  rewarding  than  the  addition 
of  a fourth,  fifth  or  sixth  degree  of  motion  freedom  to  the  cockpit. 

\ 
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g.  A platform  which  unintentionally  leaves  the  occupant  more 
than  10  - 20°  or  so  from  the  (real  or  apparent)  vertical  for  more  than 

10  - 30  seconds  or  so  will  give  rise  to  adverse  pilot  comment, 

h.  Spurious  action  cues  may  demand  more  conscious  attention 
than  meaningful  cues  of  equal  magnitude,  and  may  detract  more  from  sub- 
jective acceptance  of  a simulator  than  the  meaningful  cues  can  add. 

i.  An  impression  of  negative  g will  be  outstandingly  difficult  to 
produce.  (Ideally,  it  demands  that  the  trainee  be  quickly  inverted  at  an 
imperceptible  angular  rate!)11 


3.2. 3.3  Sensory  Thresholds  for  Motion  Perception.  The  simulation  of 
motion  must  be  achieved  by  displacing  the  training  device  very  short  dis- 
tances, but  in  a way  that  "fools"  the  trainee  into  responding  as  if  actual 
aircraft  motions  were  occurring.  In  recognizing  these  motions,  the  trainee 
is  provided  acceleration  cues  via  the  vestibular,  proprioceptive  and  tactile 
senses.  The  vestibular  system  is  the  predominant  motion- sensing  system — 
the  semicircular  canal  of  the  inner  ear  is  the  sensor  for  angular  motions; 
the  otolith  is  the  sensor  for  linear  accelerations,  as  are  also  the  proprio- 
ceptors. Since  man  is  sensitive  to  acceleration,  but  not  to  translational 
velocity,  and  since  the  simulator  is  severely  constrained  in  the  amount  of 
travel  of  displacement  that  can  be  achieved,  we  are  interested  in  man's 
acceleration  perceiving  capability.  When  does  he  sense  the  onset  of  vari- 
ous accelerations  and  under  what  conditions  do  these  threshold  values  vary? 


An  important  source  of  information  for  design  comes  from  the 
research  data  on. stimulus  thresholds  for  the  perception  of  motion.  The 
perception  of  body  motion  is  describable  in  terms  of  absolute  and  differen- 
tial thresholds.  These  fundamental  data  yield  information  and  clues  on  how, 
by  synthetic  means,  motion  perceptions  can  be  produced  indistinguishable 
from,  or  relevant  to,  those  in  flight.  These  data  are  also  involved  in  deter- 
minations of  how  man  combines  various  sensory  inputs  into  an  experience 
of  motion.  Absolute  threshold  data  are  needed  to  determine  onset  rates  of 
motion  and  washout  rates  imperceptible  to  the  trainee  after  inauguration  of 
platform  motion;  differential  threshold  data  are  needed  to  determine  ways 
of  substituting  or  integrating  different  force  components  to  achieve  the 
desired  motion  perception. 


Much  work  has  been  accomplished  in  this  century  on  establish- 
ing stimulus  thresholds  for  the  perception  of  acceleration.  As  expected, 
the  research  was  accomplished  in  a variety  of  contexts  and  for  varying 
purposes  and  certain  difficulties  emerge  in  fashioning  a system  of  values 
for  design.  The  variability  in  the  threshold  data  is  due  to  a number  of 
reasons.  Various  devices  have  been  used  in  the  studies  to  provide  the 
stimulation.  A number  of  psychophysical  methods  have  been  used  in  es- 
tablishing the  thresholds,  and  substantial  differences  exist  in  the  definition 
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of  threshold.  Considerable  variation  in  the  duration  of  the  acceleration 
force  and  latency  time  is  a characteristic  of  these  studies.  To  this  may 
be  added  the  feature  that  the  perception  of  motion  by  the  human  cannot  be 
described  in  straightforward  term^.  For  one  thing,  the  sensations  arising 
from  movement  stimulation  of  the  receptors  are  not  clear  or  precise.  The 
action  of  the  receptor  mechanisms  is  buried  in  complex  central  nervous 
processes  and  is  influenced  by  visual  and  tactile  cues.  Thus,  obtained 
measurements  of  human  thresholds  are  not  necessarily  reliable.  Also, 
man’s  sensitivity  is  considerable,  resulting  in  low  threshold  values,  thus 
making  it  difficult  to  produce  and  control  and  to  measure  the  accelerations 
at  threshold  levels. 

The  data  that  appear  most  pertinent  in  establishing  threshold 
levels  for  the  perception  of  motion  useful  to  training  device  design  are  given 
below.  Table  20  presents  relevant  research  data  on  the  absolute  human 
sensory  thresholds  for  motion  as  organized  by  Huddleston  (1966).  Unfor- 
tunately, some  questions  exist  as  to  the  accuracy  of  the  values  reported  here 
and  their  sources  (AVendt,  1967).  There  are  also  reservations  concerning 
the  value  of  these  data  for  decisions  on  design  parameters.  The  most  ob- 
vious is  that  Table  20  presents  laboratory  data  and  the  disparity  between 
laboratory  findings  and  operational  performance  is  notorious.  The  argu- 
ments as  to  why  this  is  so  are  well  known  and  need  not  be  attended  to  here  . 

At  best,  laboratory  studies  tell  us  what  the  man  will  definitely  be  unable  to 
perceive  or  utilize  in  the  operational  environment.  Thus,  the  multi- stimulus 
situation  in  the  operational  job  context  and  the  duration  and  levels  of  char- 
acteristic task  loadings  indicate  that  the  effective  thresholds  are  raised  con- 
siderably over  longer  time  samples.  Whndt^  would  raise  the  effective 
thresholds  by  at  least  a factor  of  two  and  possibly  by  a factor  of  ten  or 
more.  It  might,  for  example,  be  practical  to  use  the  maximum  values  re- 
ported in  the  literature  for  a given  threshold  area  (worst  case  assumption). 
Also,  some  data  based  on  trials  in  excess  of  one  minute  are  limited  in 
relevance  to  simulation  where  thresholds  are  based  on  only  a few  seconds 
of  time;  (the  latter  are  substantially  higher  even  though  the  f,muelder 
product11  is  apparently  an  oversimplification).  Although  laboratory  data  on 
pitch,  roll  and  yaw  thresholds  are  often  as  low  as  0.1°/  sec^,  Wendt  sug- 
gests that  it  is  more  realistic  to  accept  0.5°  to  2°  / sec^  as  the  range  of 
concern  for  motion  platform  design.  In  fact,  the  value  required  for  (almost) 
’’instant  detection11  of  angular  acceleration  can  be  estimated  as  near  5°/  sec^ 
or  higher.  (Instant  detection  based  on  exposures  under  about  0.5  sec  dura- 
tion may  be  a more  realistic  criterion  for  design  purposes  than  thresholds 
based  on  many  seconds  or  even  minutes)  Correspondingly , it  appears 

^Personal  communication.  Dr.  Hans  Wendt,  Macalester  College,  St.  Haul, 
Minnesota. 
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TABLE  20.  ABSOLUTE  HUMAN  SENSORY  THRESHOLDS 
FOR  MOTION. 


Axis 

Threshold 

Reference 

Pitch 

4°/sec  impulse  velocity 

Jones,  G.M.  , Barry,  W . 
Kowalsky,  N.  (1964)  Aerospace 
Med.  , 35:  984-989. 

Roll 

4°/ sec  impulse  velocity 

Jones,  G.M.,  Barry,  W.  &c 
Kowalsky,  N.,  (1964)  Aerospace 
Med.  , 35:  984-989. 

. . 

Mean  0.8°/sec  impulse 
velocity:  mode  1 . 6°/  sec: 

standard  range  (M  + 1SD) 
0.3-2. 1°/ sec  _ 

Benson,  A . J .,  RAF  Institute  of 
Aviation  Medicine,  personal  com- 
munication from  studies  on  N = 51 
Ss  using  sensation  cupulometry. 

Yaw 

0.035  - 4°/sec2 

Graybiel,  A.,  Kerr,  W.A.  , &c 
Bartley,  S.H.,  (1948),  Am.  J. 
Psychol.,  61:  21. 

0.12  - 0. 17°/sec2 

Clark,  B.  , & Stewart,  J.D.  , 
(1962),  Aerospace  Med.  , 33: 
1426-1432. 

0.2  - 0 . 5°/ sec2 

Johnson,  J . L . , (1959),  WADC  - 
TN-58-314,  Supp.  1 . 

0.28  - 2.0°/ sec2 
(mean  0 . 5) 

Groon,  J.J.,  &c  Jongkees,  L.B.W, 
(1948),  J.  Physiol.  , 107:  1-7. 

Down  to  0 . 2°/  sec^ 

Tumarkin,  I. A.,  (1937),  Proc  . R. 
Soc.  Med.  , 30:  599-610. 

4°/  sec  impulse  velocity 

Jones,  G.M.  , Barry,  W.  , &c 
Kowalsky,  N.,  (1964),  Aerospace 
Med.  , 35:  984-989  , 

0.06  - 0 . 035°/ sec2 

Mann,  C.W.  , & Ray,  J.  , (1956) 
USN  School  of  Aerospace  Med.  , 
NM-  00  1 - 1 10-500-41  . 

Mean  0.9°/ sec  impulse 
velocity;  mode  1.6°/ sec; 
standard  range  (M+1SD) 
0.3  - 1.7°/ sec . “ 

Benson,  A.  J.  , RAF  Institute  of 
Aviation  Medicine,  personal  com- 
munication from  studies  on  N = 103 
Ss  , using  sensation  cupulometry. 
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TABLE  20.  ABSOLUTE  HUMAN  SENSORY  THRESHOLDS 
FOR  MOTION  (Continued). 


Axis 

Threshold 

Reference 

Heave 

1.7  - 4.0  cm/s ec^ 

Gurnee,  H.  , (1934),  J.  Exp. 
Psychol.,  17:  270-285. 

1-5  cm/sec^  prone  or 
supine 

Walsh,  E.G.  , (1961),  J.  Physiol.  , 
155:  506-513. 

4-12  cm/  sec ^ 

Eogel,  E.  J.  , (1963),  Biotechnology 
p.  151,  Prentice -Hall , Englewood 
Cliffs,  N . J . 

6-15  cm/sec^  prone  or 
supine 

Jongkees,  E.B.W.,  &:  Green,  J. 
J.,  (1946),  J.  Laryngol,  61: 
529-541. 

0.003  - -02g,  dependent  on 
frequency,  for  sinusoids 
1.5-  50  c/s  ( lowest  3-6 
c/s,  highest  15  - 50  c/s) 

Goldman,  D.E.,  (1948),  US  NMRI, 
MI-004-001-1  . 

.009  - .05g,  dependent  on 
frequency,  for  sinusoids 
3-30  c/s  (lowest  8-9 
and  13-  18  c/s,  highest 
3-7  c/s) 

Gorrill,  R.  B . , &:  Snyder  , E.  W.  , 
(1957),  Boeing  Co.  D3-1189. 

.002  - l.Og,  dependent  on 
frequency,  for  sinusoids 
1 - 27  c/s  (lowest  1-9 
c/s,  highest  9 - 22  c/s) 

Parks  , D . L . , &:  Snyde  r , F . W . , 
(1961),  Boeing  Co.  D3-3512-1. 

Approx,  linear  from  0.5 
cm/sec^  at  0.5  c/s  to  20 
cm/sec^  at  100  c/s, 
sinusoids 

Dieckmann,  D . , (1958),  Ergono- 
mics, 1:  347-355. 

Sway 

1-3  cm/sec^  prone  or 
supine 

Walsh,  E.G.;,  (1961),  J.  Physiol., 
155,  506-513. 

12  - 20  cm/  sec^ 

Fogel,  L.  J.  , (1963),  Biotech- 
nology, p.  151,  Prentice -Hall , 
Englewood  Cliff s , N.J. 
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TABLE  20.  ABSOLUTE  HUMAN  SENSORY  THRESHOLDS 
FOR  MOTION  (Continued). 


Axis  Threshold  Reference 


Sway  Linear  from  0.00  3 cm 

double  amplitude  at  4 c/s 
to  0.000  3 err.  at  40  c/s, 
for  sinusoids,  supine 


Fogel,  L.J.,  (1963),  Biotechnology, 
p.  151,  Prentice-Hall,  Englewood 
Cliffs,  N.J. 

0.5  - 1.3  cm/sec^,  Ishimoto , M.  (1932),  Bull.  Earth- 

dependent  on  frequency,  quake  Res.  Inst.  , Vol.  10,  Tokyo, 

for  sinusoids  0.5  - 10 
c/ s (lowest  at  3 - 5 c/s, 
highest  at  8 - 10  c/s) 

0.6  - 2.0  cm/sec^,  Ishimoto,  M.  , Ootsuka,  O.  , (19  33), 

dependent  on  frequency.  Bull.  Earthquake  Res  . Inst.  , Vol. 

for  sinusoids  0.4-10  11,  Tokyo, 

c/s  (lowest  at  1 - 4 c/s, 
highest  at  6 - 10  c/s) 

Linear  from  0 . 002  Reiher,  H.,  & Meister,  F.J.  , (1931), 

cm/sec^  double  ampli-  Forsch,  2:  381-386,  Berlin." 

tude  at  5 c/s  to  0.0002 

cm/sec^  at  60  c/s  for 

sinusoids 

Approx,  linear  from  Dieckmann,  D.  , (1958)  , Ergonomics  , 

0 . 2 cm/  sec  ^ at  0.2  c/s  1:  347  -355  . 

to  TO  cm/  sec ^ at  50 
c/s,  for  sinusoids 


(from  Huddleston,  1966) 


Surge  12  - 20  cm/  sec' 


Reiher,  H.  , & Meister,  F.  J.  , (1931), 
Forsch.,  2:  38  1-386,  Berlin. 
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justified  to  use  values  such  as  0.3°/sec^  as  in  effect,  n subliminal"  since 
also,  fadeback  accelerations  compatible  with  this  value  have  been  found 
practical  by  a manufacturer  of  simulation  equipment. 

Huddleston  lists  heave,  sway  and  surge  thresholds  in  a range 
of  0 . 5 to  20  cm/sec^  (corresponding  to  approximately  0.0005  to  0,02  g)  . 
Wendt  feels  these  values  are  rare  and  points  to  other  studies  where  values 
up  to  0.08  g or  higher  were  obtained.  The  range  of  values  reported  differ 
by  about  two  orders  of  magnitude  (which  is  not  uncommon  in  attempts  to 
determine  thresholds).  For  operational  environments  , Wendt  recommends 
linear  accelerations  in  excess  of  about  0.1  g if  reliable  cue  value  is 
essential. 


Clark  (19  67)  has  also  summarized  a number  of  studies  which 
report  stimulus  thresholds  for  the  perception  of  angular  acceleration. 

Table  21  presents  his  summary. 

The  stimulus  thresholds  reported  for  angular  accelerations  vary 
considerably  between  0.035°/  sec^  and  8.2°  / sec^  with  a median  around 
1.0°  / sec^.  Thresholds  are  lower  about  the  yaw  axis  than  about  the  pitch 
and  roll  axes.  Clark  and  Stewart  (1962)  in  the  study  of  pilot  perception  of 
angular  acceleration  about  the  yaw  axis  at  NASA  Ames  Research  Center  , 
obtained  values  as  low  as  0. 13°/sec^  for  latency  times  of  8-14  seconds; 
and  about  1°/  sec^  for  a latency  time  of  2 seconds.  For  mean  reaction 
latency  times  of  from  0.5  to  1.0  seconds  the  acceleration  threshold  was 
10.0^/sec^.  Doty  (1969),  working  in  the  same  laboratory,  measured 
0.  1°  / sec^  for  6 seconds  stimulus  duration. 


The  highest  value  (about  the  pitch  axis)  was  obtained  from  a 
pilot  operating  a Link  Trainer  (Sadoff,  Matteson  and  Havill,  1955).  The 
great  variation  in  values  is  to  be  expected,  considering  the  types  of  tasks 
involved,  the  definitions  of  threshold  used  and  the  duration  of  the  angular 
accelerations  employed  (from  less  than  a second  to  more  than  a minute)  . 
Direct  comparisons  among  studies  are  hampered  in  particular  by  the  varia- 
tions in  duration  of  acceleration.  It  was  once  believed  that,  uniformly, 
acceleration  X time  = a constant  within  certain  limits  , at  or  near  the  thresh- 
old (the  "Muelder  product”)  (Muelder,  1908;  Van  Egmond,  et  al,  1949). 

The  product,  according  to  some  experiments,  increases  for  both  longer  and 
shorter  time  durations.  However,  certain  earlier  assumptions  underlying 
the  use  of  the  Muelder  product  (cf.  Muelder,  1908)  are  not  as  generally 
valid  as  once  thought.  For  example,  in  a recent  study  by  Doty  (1969)  the 
ocolugyral  illusion  was  used  to  test  the  reliability  of  the  integration  phenom- 
enon that  is  ordinarily  postulated.  The  illusion  in  question  tends  to  be  very 
stable  across  and  within  individuals,  and  is  probably  the  most  sensitive 
indicator  among  several  that  can  be  used  to  measure  thresholds  for  angular 
acceleration.  In  the  Doty  study,  average  thresholds  were  low,  ranging 
from  0. 10°/  sec^  to  0. 62°/sec^  depending  on  stimulus  duration.  However, 
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the  acceleration  x time  products  were  not  constant.  Rather,  they  range 
from  approximately  0.3°/ sec  (measured  for  a stimulus  duration  o 
sec)  to  approximately  0.6°/sec,  the  latter  measured^  6 . 0 sec -duration 
of  exposure  Graphic  extrapolation  of  these  data  leads  to  about  0.8  /sec 
as  an^ estimate  of  the  acceleration  x time  product  near 

of  10  sec,  etc.  In  other  words,  while  the  function  appears  to  be  a li 
one,  the  slope  is  not  zero,  but  is  significantly  positive. 

• - - - A summary  of  the  results  of  laboratory  data  on  determining 

thresholds  for  angular  acceleration  is  shown  in  Figure  31  The  da.a  S 
that  man  relates  angular  acceleration  to  reaction  latency  (time).  For  very 
low  angular  acceleration  the  time  lag  prior  to  detection  is  SIS™*IC  ’ 
e g , for  accelrations  of  0.05  to  0.1°/ sec2  there  is  a substantial  lag 
(8  10  seconds)  in  sensing  acceleration.  For  accelerations  in  excess  of 

10O/iec2 . the  latency  is  on  the  order  of  1/2  second  prior  to  detectron. 
Figure  32  provides  threshold  data  on  linear  acceleration. 

Wendt  (1966)  in  a review  of  the  literature  on  threshold  values 
has  selected  excerpts  relevant  to  the  design  of  moving  base  simulators. 
Among  his  conclusions  are  the  following: 

There  is  evidence  that  sensitivity  is  greatest  for 
rotation  around  the  pitch  axis,  less  marked  for 
the  yaw  axis  and  least  for  the  roll  axis.  The 
differentials,  however,  appear  to  be  too  small 
to  serve  as  useful  guidelines  for  design. 

. Natural  or  frequently  experienced  cue  levels  are 

of  value  in  that  a pilot  knows  what  certain  £ values 
feel  like  and  these  should  be  emphasized  in  that 
they  may  serve  as  a frame  of  reference  for  motion 
even  though  other  j*  levels  are  not  simulated  with 
similar  fidelity.  One  such  well-known  standard 
criterion  is  the  self-imposed  vertical  accelera- 
tions arising  from  stooping,  standing  up  from  the 

seated  position,  etc.  This  component  ranges  . from 

0 0 3 to  0.40  g with  a mean  of  0.15  (measur edtjpn 
military  popillations)  . Thus  , a pilot  not  familiar 
with  the  aircraft  may  conceivably  be  able  to  com- 
pare certain  accelerations  with  those  familiar  to 

him. 

■ The  direction  of  linear  acceleration  can  be  accu- 

rately discriminated  for  values  of  >10  cm/ s ec 

when  force  is  alternately  applied  in  opposite 

directions  . This  corresponds  to  >0.01  g_  and  is 

lower  than  values  obtained  when  simple  linear 

q accelerations  are  discriminated  from  zero. 
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Figure  32.  Latency  times  for  perception  of  acceleration  along 
the  sagittal  axis  with  the  head  upright. 

(from  Clausen,  et  al,  1968;  based  on  data  from  Meiry,  1965) 
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• Vertical  motion  can  be  used  to  simulate  pitch  when 
the  pivot  point  is  some  distance  from  the  man  (e.g.  , 
30  feet  behind  and  where  the  vertical  position  of  the 
cab  is  washed  out  to  its  starting  position).  Vertical 
displacement  with  washout  is  more  satisfactory  than 
scaling  down  the  p'tch  axis  distance  by  factors  vary- 
ing from  2 to  5. 


• Angular  acceleration  thresholds --laboratory  data 
range  from  0.2°/sec2  to  2.0°/ sec2  (assuming  long 
latency  periods).  When  response  latency  is  the 
criterion,  about  2 seconds  are  required  to  detect 
3.0°/ sec2;  3 seconds  are  required  to  detect  2.0°/ 
sec2;  nearly  7 seconds  are  required  to  detect 
1.0°/sec2.  In  terms  of  short  time  duration,  as 
is  the  case  in  simulation,  man  is  relatively  insensi- 
tive to  changes  in  angular  motion  in  the  absence  of 
other  cues  . Some  data  suggest  that,  if  the  maxi- 
mum permitted  response  latency  is  held  to  about 
1.5  seconds,  and  if  latency  is  taken  as  the  chief 
criterion  of  threshold,  then  the  average  observer 
carvr  A tell  4°/sec2  from  10°  - 15°/sec2  accelerations. 

3.2. 3.4  ActJe  vine  Acceleration  Cues  in  Simulation.  The  physical  dis- 
placement limitation j places  serious  constraints  on  both  acceleration  and 
frequency  response.  Displacement  varies  inversely  as  the  square  of  the 
frequency  for  sinusoidal  motion;  the  higher  the  frequency  of  the  accelera- 
tion the  smaller  the  displacement.  As  frequencies  fall  below  one  cycle  per 
second  for  a ;wen  acceleration  level,  displacement  requirements  increase 
considerably  For  example,  an  acceleration  of  _+  \g  at  1 cps  requires  _+10 
inches  displacement  while  an  acceleration  of  _+  l_g  at  0 . 1 cps  requires  +85 
feet  (Graham,  1968).  A plot  of  frequency  vs.  displacement  for  various 
accelerations  is  shown  in  Figure.  33.  Representing  the  low  frequency  or 
sustained  accelerations  is  particularly  difficult. 


Because  of  the  displacement  limitations,  techniques  are  required 
for  deceiving  the  human  by  providing  cues  that  have  signal  value  and  have 
relevance  (meaning)  to  the  real  world  counterpart.  Several  techniques  of 
simulating  g forces  are  currently  practiced  in  simulator  design.  These  are; 
the  use  of  onset  cues  and  washout  acceleration;  scaling  of  acceleration;  and 
the  use  of  signaling  techniques  such  as  inflatable  seat  cushions  and  seat  belt 
tighteners.  Each  is  discussoct  below. 


3.2. 3.4.  1 Acceleration  Onset  and  Washout.  Acceleration  and  discrim- 

ination of  differences  in  accelerations  is  3.  function  of  the  rate  of  onset  of 
acceleration  (jerk  or  jolt).  Matheny,  et  al,  (1963)  hold  that  rate  of  onset 
may  be  more  of  a direct  cue.  than  any  absolute^yalqe  of  acceleration. 

Rate  of  onset  has  some  threshold  value  and  pr0deetes  the  acceleration  in 
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Equation  for  Sinusoidal  Motion 
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Single  Amplitude  Displacement  - Feet 

Figure  33.  Displacement  Versus  Frequency  for  Various 
Cyclic  Accelerations  (from  Graham.,  1968) 
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time.  To  put  the  simulator  into  motion,  "requires  that  it  be  brought  to  a 
certain  rate  by  an  acceleration  and  to  this  acceleration  by  a certain  rate 
of  onset  of  acceleration,  etc.  , each  of  these  successive  derivatives  having 
occurred  successively  earlier  in  time.  . . If  the  operator  can  sense  this 
third  derivative  of  position  (jerk)  proprioceptively  and  initiate  even  an 
approximate  corrective  response,  he  is  considerably  ahead  of  the  visual 
sense  alone  in  controlling  the  system.  11  Less  simulator  displacement  is 
required  for  the  above -threshold  value  for  rates  of  onset  than  is  for 
accelerations  . 


Thus,  the  onset  of  the  _g  force  is  presented,  then  the  force 
is  quickly  faded  away  and  the  simulator  returns  to  the  neutral  position 
without  the  man  being  aware  of  the  washout  of  the  force.  Only  a relatively 
small  motion  of  the  platform  is  required. 

Wendt  (1966)  states  that  some  laboratory  studies  yield  "sub- 
liminal acceleration"  ranging  from  about  0.3°  to  0-5°/  sec^.  These  are 
conservative  (safe)  and  it  is  probable  that  washout  levels  in  current  simu- 
lators slightly  less  or  of  comparable  orders  of  magnitude  have  been  found 
acceptable.  In  practice,  0.  lg  washout  accelerations  for  linear  motion  have 
also  been  employed  in  simulator  design  with  some  success. 

The  Link  group  (Flexman,  et  al,  1968)  emphasize  that  the 
acceleration  involved  in  washout  should  be  subliminal,  so  as  not  to  provide 
a false  cue,  but  still  remain  as  rapid  as  possible.  Since  for  angular  motion 
the  product  of  latency  time  (time  required  to  detect  an  acceleration)  and 
the  acceleration  itself  (the  "Muelder  product")  is  approximately  constant 
for  latencies  less  than  2.5  seconds  (but  cLthe  qualifications  stated  earlier), 
the  specification  of  fading  acceleration  is  not  a matter  of  setting  a single 
acceleration  limit.  A higher  fading  acceleration  can  be  tolerated  when 
washing  out  a small  angle  than  when  washing  out  a large  one,  since  the 
latter  requires  more  time.  Thus,  while  it  would  be  safe  to  use  an  accel- 
eration limit  based  upon  a maximum  excursion,  this  would  result  in  need- 
lessly slow  returns  from  excursions  less  than  the  maximum.  In  additiop., 
it  is  argued  there  that  the  thresholds  obtained  from  laboratory  data  should 
be  multiplied  by  a factor  of  2 or  3 to  account  for  the  decreased  sensitivity 
associated  with  task  loading.  The  problem  of  washing  out  the  velocity  re- 
sulting from  the  acceleration  while  the  trainee  still  continues  to  sense 
motion  subjectively  is  crucial  for  design.  A single  cycle  involves  the  fol- 
lowing: onset  of  acceleration  (jerk);  the  period  of  time  taken  by  the  velocity 

washout;  and  the  time  required  to  return  the  system  to  its  zero  position 
which  involves  both  a subliminal  acceleration  and  deceleration;  This  cycle 
is  involved  each  time  motion  is  generated. 

The  washout  threshold  has  not  received  enough  careful 
attention  in  simulator  design.  Wendt^  says  it  has  been  ignored  or  left 

Hans  W.  Wendt,  personal  communication. 
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to  incidental  system  characteristics  whose  parameters  are  not  fully  known. 
As  a consequence,  some  commercial  simulator  users  have  found  it  prac- 
tical to  reduce  motion  system  gain  or  even  eliminate  degrees  of  freedom 
along  certain  axes  because  of  cue  conflict  or  potential  negative  transfer 
incurred  under  such  unfavorable  circumstances  which  had  not  been 
anticipated. 


Graham  (1968)  states  that  in  practice,  washout  motions  in- 
trude considerably  above  the  detection  threshold  and  not  much  can  be  done 
about  it,  since  humans  are  too  sensitive  to  acceleration.  The  use  of  wash- 
out apparently  incurs  poor  phase  relationships  among  acceleration,  vel- 
ocity and  displacement  cues.  Washout  works  best  when  the  simulation  as 
a whole  is  unrealistic.  When  the  simulation  is  realistic,  washout  gener- 
ates some  confusion.  The  reason  for  this  conjecture  is  that  with  unreal- 
istic simulation,  the  man  does  not  attend  to  all  cues  since  he  is  aware 
that  some  are  false,  whereas  in  the  realistic  situation  he  attends  to  all 
cues  and  the  erroneous  washout  cues  make  for  confusion. 

3. 2. 3. 4. 2 Scaling  of  Motion  Forces.  An  option  for  design  is  partial 

force  simulation  involving  a reduction  in  the  dynamic  range  required  of 
the  motion  platform.  This  calls  for  a scaling  down  of  the  cue  forces  from 
the  correct  amplitudes  without  affecting  the  functional  relationships  of  the 
simulator  in  its  operation.  The  obvious  objection  to  this  is  that  it  degrades 
fidelity  significantly.  However,  it  may  be  argued  that  little  functional 
difference  would  result  since  large  inertial  forces  are  not  differentiated 
well  (quantitatively)  by  man.  In  the  visual  motion  context,  these  cues 
would  tend  to  only  inform  him  of  the  order  or  magnitude  of  the  force  until 
the  visual  cues  become  dominant.  If  this  is  true,  then  scaled  down  forces 
represent  a functional  similarity  to  actual  motion  forces  provided  they  are: 
of  the  correct  sense;  inform  the  pilot  of  rough  changes  in  magnitude;  and 
do  not  incur  cue  conflicts  . The  latter  point  is  important.  Angular  forces 
can  be  represented  quite  realistically  at  least  more  so  than  the  transla- 
tional forces.  Now,  if  the  key  issue  is  conspicuous  cue  value,  and  if  trans- 
lations (surge,  heave,  etc.)  usually  occurred  independently  of  rotation, 
then  scaling  would  appear  feasible.  Under  these  circumstances  a factor 
of  up  to  5 or  even  higher  would  be  worth  exploring.  On  the  other  hand,  if 
the  device  to  be  simulated  implies  frequent  compounding  of  linear  and 
angular  vectors,  as  in  some  maneuvers  of  high  performance  aircraft, 
then  a combination  of  non-scaled  angular  forces  with  scaled  linear  ones 
may  result  in  cue  conflict  and  potential  negative  transfer.  Intuitively  it 
would  seem  that  ranges  can  be  found  where  satisfactory  cue  value  and 
some  positive  transfer  (or  at  least,  absence  of  significant  negative  .trans- 
fer) can  be  obtained  from  a scaling  down  of  both  translational  and  rota- 
tional accelerations  even  though  simulator  mechanics  might  permit  higher 
and  near-realistic  formulas  for  certain  rotational  degrees  of  freedom. 
Unfortunately,  the  situation  is  further  complicated  by  the  observation  that 
--aside  from  absolute  thresholds  - -the  differential  thresholds  for  linear 
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and  angular  motion  appear  to  be  quite  different  although  research  in  this 
area  has  been  rare.  In  practical  terms  , a 25  percent  reduction  in  linear 
acceleration,  for  example,  may  be  substantially  different,  subjectively, 
from  a 25  percent  reduction  in  a rotational  component. 

Graham  (19  68)  states  that  scaling  down  has  already  been 
used  successfully  in  the  situation  where  pilots  flew  a representative  air- 
craft prior  to  using  the  simulator.  Using  the  Cooper  Rating  Scale  for 
system  evaluation,  they  were  able  to  extrapolate  the  reduced  acceleration 
cues.  When  a pilot  experienced  some  difficulty  in  the  simulator  he  cou’d 
mentally  multiply  his  problem  and  rate  it  accordingly.  The  use  of  objective 
performance  measures  (e.g.  , tracking  error)  could  give  erroneous  results, 
however,  in  the  scaled,  down  situation. 

3.2. 3.4.  3 Tilting  of  the  Cockpit  or  Body.  Tilting  of  the  cockpit  or  the 

individual  to  resolve  the  normal  gravity  vector  into  a continuing  subjective 
vertical  vector  and  the  equivalent  of  a longitudinal  or  lateral  acceleration 
is  another  technique  of  some  promise  particularly  when  used  in  conjunction 
with,  customary  motion  systems.  For  exumple , pitch  up  of  the  cockpit  can 
be  ^sed  to  provide  a cue  for  forward  acceleration;  bank  can  be  used  to  pro- 
vide a cue  for  lateral  motion  (assuming  a compatibility  between  visual  and 
motion  cues). 


Wendt  and  Cohen-*-  state  that  the  tilt  mode  when  added  to  the 
motion  system  for  a commercial  aircraft  simulator  has  produced  equivocal 
results.  Pilots  have  reported  conflicting  and  unrealistic  sensations,  even 
a feeling  of  sliding  off  the  seat,  etc.  The  writers  believe,  however,  that 
one  or  more  of  the  constraints  inherent  in  the  tilting  technique  have  been 
violated,  and  in  particular,  there  has  been  a failure  to  consider  rotational 
acceleration  thresholds.  They  list  four  conditions  that  must  be  met  to 
assure  the  feasibility  of  the  tilting  technique: 


a.  Longitudinal  or  lateral  G is  above  threshold  to  provide 
distinctive  cue  function. 

b.  The  perceived  G along  the  body  vertical  is  indistinguish- 
able from  the  normal  1 _g  level,  to  avoid  the  sensation  of  downward  accel- 
eration (falling). 


c _ The  reduction  of  G along  the  body  vertical  is  small 
enough  to  avoid  the  "sliding  off  the  Teat"  effect  in  conjunction  with  the 
horizontal  force  component  (the  geometry  of  conditions  a + b set  the 
upper  limit  for  any  simulation  of  horizontal  acceleration  that  can  be 
achieved)  . 


IWendt,  H . W . and  Cohen,  E.  Simulating  Lateral  and  Longitudinal  Accel- 
erations  by  Subliminal  Tilting  of  the  Operator  Vertical.  Link  Group 
Engineering  Report,  Binghamton,  New  York,  1969*  (Pre-publication  draft) 
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^ ■ The  rotation  of  the  body  through  pitch  or  bank  occurs  in 
such  a way  that  the  angular  acceleration  and  deceleration  of  this  auxiliary 
motion  remains  subliminal. 


The  tilting  principle  includes,  unfortunately,  incidental  horizontal  and  ver- 
tical accelerations  which  interact  with  the  buildup  of  the  intended  G state, 
but  Wendt  and  Cohen  calculate  from  available  threshold  data  and  the  geom- 
etry of  the  human  body,  that  their  magnitude  is  not  large  enough  to  interfere 
with  the  intended  simulation  of  longitudinal  and  lateral  G.  Generally,  the 
feasibility  of  the  tilting  technique  is  determined  by  the  angular  acceleration 
thresholds  of  the  man  in  the  plane  of  tilting  and  by  the  extent  to  which  loss 
of  G along  the  vertical  is  subjectively  perceived  (with  or  without  other  s'de 
effects).  The  limiting  angular  motion  thresholds  are  determined  jointly  by 
exposure  duration  and  magnitude  although  their  product  is  a constant  only 
in  gross  approximation.  Based  on  the  available  sensory  threshold  data  and 
a number  of  realistic  assumptions,  the  authors  conclude  that  the  absolute 
minimum  delay  is  estimated  at  about  1.7  seconds  longitudinal  and  1.9 
seconds  lateral.  In  other  words,  a minimum  of  1 . 7 seconds  would  be  re- 
quired from  the  beginning  of  the  tilting  to  first  sense  a forward  acceleration 
(at  or  slightly  above  threshold)  and  yet  not  sense  the  rotation  which  is  a 
necessary  part  of  this  simulation  technique  . For  a simulated  level  of  0.2g 
longitudinal,  transition  to  the  endpoint  requires  from  4 to  9 seconds  (de- 
pending  on  details  of  the  tilting  program  used).  The  upper  feasible  limit 
for  simulation  seems  to  be  between  0 . 3 £ and  0.4  g,  and  indirectly  deter- 
mined by  distortions  of  the  vertical  G vector.  For  0.4  g,  the  minimum 
theoretical  delay  is  calculated  as  7.8  seconds  for  the  longitudinal  acceler- 
ation mode  simulated  by  pitch  angle.  So  far  as  the  maximum  rate  of  change 
G that  can  be  simulated  is  concerned,  similar  calculations  lead  to  an 
order  of  0.05  g/ second. 


It  appears  that  the  tilting  technique  should  provide  ade- 
quate fidelity  when  combined  with  a two  or  a three  degree  of  fireedom 
motion  platform.  The  tilting  would  simulate--in  some  instances  fully  so-- 
the  required  sustained  and  more  stable  G components;  the  conventional 
platform  would  reproduce  higher  frequency  vertical  and  longitudinal  or 
lateral  motion.  Wendt  and  Cohen  state  that  the  tilting  principle  is  not 
applicable  in  vehicle  simulation  when  normal  maneuvers  involve  absolute 
G loads  in  excess  of  0.5  to  0.6,  or  for  loads  whose  final  sustained  magni- 
tude is  reached  at  rates  in  excess  of  0.4  to  0.5  g/ second. 


j.  2.  3.4.4  Differential  Pressures  on  the  Body  as  Motion  Cues.  Some 

exploratory  work  has  been  devoted  to  producing  motion  cues  by  stimulat- 
es portions  of  the  body.  These  include  controlled  pressure  redistributions 
by  means  of  inflatable  seats  and  backrests,  by  altering  tensions  on  seat 
belts  / shoulder  harnesses,  and  by  means  of  duplicating  the  shearing  forces' 
at  the  major  support  points  of  the  human  which  accompany  actual 
accelerations. 
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Inflatable  seat  cushions  and  seat  belt/  shoulder  harness 
tension  devices  (for  vertical  or  longitudinal  G)  serve  to  signal  the  pilot 
that  accelerations  are  occurring.  As  such  they  may  serve  most  usefully 
when  supplemented  by  brief  physical  motion  cues  supported  by  relevant 
visual  cues.  To  simulate  sustained  longitudinal  acceleration  (e.g.  , takeoff 
roll),  the  sequence  could  be  as  follows:  the  acceleration  onset  (jerk),  which 

is  then  washed  out,  coupled  with  pressures  on  the  harness"  or  against  the 
pilot*s  back  plus  the  visual  cues  of  increasing  speed  along  the  runway  (and 
tilting  of  the  pilot  vertical). 

A pneumatic  seat  system  made  up  of  six  differentially  in- 
flatable sections  was  developed  for  providing  subjectively  valid  motion  cues 
while  not  moving  the  cockpit  (Barrett,  et  al,  1969).  The  seat,  via  com- 
puter controlled  air  pressure,  produced  buffeting  effects,  small  displace- 
ments (0-20  Hz  range),  acceleration,  long  term  coordinated  and  uncoordi- 
nated turns  , pullouts  and  pushovers  and  deceleration  effects  simulating  a 
landing  approach.  It  also  simulated  turbulence  and  general  background 
vibration.  The  seat  motion  significantly  increased  realism  for  a series  of 
six  maneuvers  flown  by  eight  general  aviation  pilots.  When  the  pilots, 
blindfolded,  were  presented  the  motion  cues,  they  identified  correctly  the 
maneuvers  at  an  85%  average. 

Powe,  et  al  (1963),  employing  a segmented  air  cushion  for 
producing  differential  pressure  effects  reported  that  the  seat  magnified 
the  amount  of  tilt  by  a factor  of  2.5.  The  Link  group,  General  Precision 
Systems,  has  also  developed  a technique  for  partially  simulating  accelera- 
tion cues  with  the  design  of  a harness  system  involving  tension  adjustments. 

These  exploratory  studies  exemplify  the  attempt  to  induce 
the  perception  of  moaon  by  pressures  on  the  human  body  instead  of  moving 
the  aircraft  cockpit.  The  economies  of  this  approach  for  training,  how- 
ever, have  not  been  verified  nor  are  sufficient  data  available  for  compari- 
sons with  other  techniques,  or  to  justify  the  approach  as  a means  of  aug- 
menting physical  motion  in  the  simulator  except  on  theoretical  and  rational 
grounds . 

3. 2. 3. 4.  5 Shearing  Forces.  One  study*  investigated  the  sensations 

resulting  from  major  shearing  forces  arising  at  the  man/ support  inter- 
faces. The  forces  of  concern  are  described  below.  Figure  34  depicts 
the  pressure  points  for  a linear  accelerating  vehicle;  Figure  35  depicts 
the  pressure  points  when  the  vehicle  makes  a large  radius  turn  to  the 
right  (lateral  acceleration).  In  this  study,  12  blindfolded  females  were 

* Wendt , H.W.,  Farley,  W.R.,  Bergwall,  U.W.  and  Bucher,  B.G. 

Acceleration  feel  as  a function  of  acceleration.  Unpublished  study. 

Valparaiso  University,  Department  of  Psychology,  May,  1968. 
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Figure  34.  Sensations  in  Vehicle  Linear  Acceleration. 
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Pressure  or  contact  Neck  muscles  strain 

on  left  elbow,  no  to  keep  head  erect, 

contact  on  right. 
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Figure  35.  Sensations  for  a Large  Radius  Turn  to  the  Right  \or  Lateral  Accelerations) 
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subjected  to  acceleration,  deceleration  and  lateral  acceleration  in  the 
realistic  environment  of  a moving  automobile  . Acceleration  was  held  to 
0.15  g under  all  conditions,  and  the  median  duration  of  exposure  was 
7 seconds.  The  subjects  were  instructed  to  observe  closely  their  sensa- 
tions and  to  indicate  their  order  of  appearance  as  well  as  relative  promin- 
ence, The  purpose  was  to  study  the  role  that  normal  pressure  as  well  as 
shear  forces  would  play  in  the  subjective  experience  so  that,  in  later  sys- 
tematic attempts  (and  hopefully,  using  a more  versatile  motion  platform) 
the  "natural11  cues  available  to  an  operator  undergoing  acceleration  could 
be  synthesized.  For  example,  a shear  force  normally  apparent,  and  often 
conspicuous,  in  the  shoulder  region  during  lateral  acceleration  can  be 
duplicated  by  keeping  the  operator  torso  stationary  while  sliding  a movable 
portion  of  the  backrest  against  his  shoulder  blades.  The  chief  results 
were  as  follows: 

a.  Generally,  sensations  are  substantially  more  prominent 
in  the  upper  half  of  the  body  than  below  the  waistline. 

b.  The  rank  orders  obtained  for  "appearance"  and  "promi- 
nence" are  essentially  the  same.  To  the  extent  that  differences  should  be 
found  in  subsequent  studies,  it  is  believed  that  for  purposes  of  simulation, 
the  ordering  in  time  should  be  given  more  consideration  than  "prominence." 

c.  The  rank  orders  differ  somewhat  for  the  three  conditions. 
This  is  taken  to  indicate  that  simulation  can  capitalize  on  different  patterns 
as  well  as  on  the  appearance  or  non-appearance  of  a given  cue  type.  The 
following  rank  orders  were  obtained  for  the  three  most  conspicuous  areas 
for  each  experimental  condition: 

Neck/ head  This  cue  almost  uniformly 

emerges  as  the  first  one 
noticed  by  all  subjects;  it 
appears  to  be  substantially 
more  important  than  the 
? other  two . 

Shoulde  r / back 

Buttocks 


Forward  Shoulder /back 

deceleration 

(braking,  etc.)  Neck/head 
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Forward  Hands,  feet 

deceleration 

(braking,  etc.) 

(continued) 


Shoulder /back  This  cue  region  appears 

to  be  much  more  important 
than  the  ones  following. 

Thighs 
Elbows 

3.2.4  Some  Specifications  for  Achieving  Motion.  It  is  obvious  that  gen- 
eral prescriptions  for  incorporating  motion  into  the  design  of  ground-based 
trainers  do  not  exist.  There  is  no  MstandardM  set  of  data  for  the  design  of 
motion  systems  for  simulators  and  the  data,  available  are  not  sufficient  as 
a body  of  knowledge  to  serve  as  guidelines  on  the  kinds  and  amount  of 
motion  displacement  required  for  simulating  motion  cues.  The  lack  of 
standard  guidelines  to  design  is  compounded  by  the  feature  that  motion 
systems  must  be  "tailored11  to  the  training  tasks  specified  for  the  simulator 
and  to  the  characteristics  of  the  aircraft  involved,  since  motion  cues  are 
dependent  on  the  aircraft  type  being  simulated.  Task  analysis  is  required 
(see  Section  II  of  this  report)  in  order  to  specify  those  motion  parameters 
and  values  most  suited  to  the  training  purpose  of  the  device.  Thus,  design 
practices  today  rely  more  on  empirical  solutions  rather  than  on  precise 
quantitative  information;  definitions  of  motion  requirements  tend  to  be 
subjective  based  primarily  upon  operating  practices  {including  best  esti- 
mates , combinations  of  research  data  and  logic,  and  "on-site n tryouts 
conducted  by  professionals  from  the  user  agency). 

The  specifications  for  the  simulated  motion  system  is  based  on 
the  design  characteristics  of  the  given  aircraft  type.  The  current  practice 
is  mirrored  in  the  Military  Specifications  (MIL-T  8235)  and  MIL- T 92  1 2B 
(USAF)  for  Flight  Trainer s . The  general  mption  requirements  for  simu- 
lators are  exerpted  below.  An  example  of  a specific  motion  system  re- 
quirement (fbr  Device  23324  Synthetic  Flight  Training  System’®')  is  also 
excerpted.  ; 


I : 

1 Specification  for  Synthetic  Flight  Training  System,  Engineering  Develop- 
ment Model  Device  2B24,  353-514  Task,  1955,  U.S.  Naval  Training 
Device  Center  , Orlando , Florida  , 13  September  1968  . 


Lateral 
acceleration 
(wide  circle  turn 
or  other) 


In  terms  of  inter-subject 
variability  the  order  here 
is  less  clearcut  than  in 
either  forward  or  lateral 
acceleration . 
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• Motion  simulation.  When  specified  in  the  detail  trainer  speci- 
fication, the  cockpit  shall  be  capable  of  limited  motion  to  pro- 
duce kinesthetic  sensations  representative  of  those  experienced 
in  the  design  basis  aircraft  in  the  areas  specified  herein.  The 
following  limits  of  motion  from  the  neutral  altitude  shall  apply: 

(a)  Angular  rotation  in  roll  plus  or  minus  15  degrees 

(b)  Angular  rotation  in  pitch  plus  or  minus  15  degrees 

(c)  Vertical  translation  at  the 

pilot  position  plus  or  minus  6 inches 

- Motion  resulting  from  flight  maneuvers.  Within  the  avail- 
able limits  of  motion,  the  cockpit  shall  respond  to  flight 
control  inputs  by  motion,  in  the  appropriate  angular  or 
translational  sense,  so  as  to  simulate  the  onset  of  accel- 
eration forces  associated  with  the  resulting  maneuver  up 

to  an  increment  of  at  least  0 . 5g  force.  Following  the  initial 
transient  g force,  (a)  for  maneuvers  which  result  in  unaccel- 
erated flight,  the  cockpit  attitude  shall  correspond  to  the 
flight  attitude,  and  (b)  for  maneuvers  which  result  in  posi- 
tive accelerated  flight,  the  cockpit  shall  return  slowly  and 
imperceptibly  to  normal  straight  and  level  flight  attitude. 

The  aircraft  instrument  indications  shall  reflect  the  actual 
simulated  flight  condition  regardless  of  cockpit  attitude. 

- Oscillations  and  vibrations.  Characteristic  periodic  oscil- 
lations and  vibrations  shall  be  qualitatively  simulated  by  the 
application  of  frequencies,  amplitudes,  and  directions  repre- 
sentative of  those  in  the  design  basis  aircraft. 

- Background  motion.  A random,  low-frequency,  low- 
amplitude,  multidirectional  cockpit  oscillation  with  repeti- 
tion cycles  of  not  less  than  15  minutes  duration  shall  be 
provided  to  simulate  the  effect  of  being  in  other  than  a 
stationary  platform.  Such  simulation  shall  be  controllable 
in  intensity  so  as  to  simulate  varying  degrees  of  rough  air. 
Instrument  indications  shall  be  qualitatively  affected  by 
rough  air . 


Motion  system  operation  and  control.  The  motion  system 
shall  operate  smoothly  without  hunting  or  snubbing  against 
fixed  stops,  in  such  a manner  as  not  to  cause  damage  to 
equipment  or  injury  or  unrealistic  discomfort  to  trainees 
or  other  personnel.  Overall  system  energization  and  acti- 
vation and  each  mode  of  operation  shall  be  separately 
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controllable  by  the  instructor  with  appropriate  indication 
of  the  mode  of  operation  and  power  energization.  Energiza- 
tion or  activation  shall  not  be  possible  without  positive  arm- 
ing action  by  and  indication  to  the  trainee  seated  in  the 
cockpit.  It  shall  also  be  possible  for  the  trainee,  by  easily 
accessible  control,  to  completely  deactivate  the  system  at 
any  time.  Initial  system  activation  and  energization  as  well 
as  deactivation  or  deenergization  by  whatever  means  accom- 
plished shall  be  smooth  and  gradual  with  no  sudden  or  abrupt 
motion  imparted  to  any  part  of  the  system  structure.  When- 
ever the  system  is  or  becomes  deactivated,  it  shall  remain 
or  return  to  a neutral  level  position.  When  it  is  completely 
deenergized,  it  shall  be  seated  in  the  neutral,  level  position 
on  a suitable  supporting  cradle  or  framework  so  that  all 
actuating  elements  are  unloaded  and  operation  of  the  motion 
system  is  prevented.  In  case  of  any  motion  of  the  cockpit 
beyond  its  normal  operating  limits,  or  of  any  failure  or 
malfunction,  in  the  motion  actuating  system  itself  or  in  any 
other  trainer  system  that  might  result  in  the  introduction  of 
spurious  signals  into  the  actuators,  the  motion  system  shall 
automatically  deactivate.  When  such  failure  involves  loss  of 
the  motion  system  power  source,  return  to  a neutral  posi- 
tion shall  not  be  required. 

MIL- T- 92  12 B (USAF): 

Motion  system  limits.  Unless  otherwise  specified  in  the  detaU 
specification,  the  motion  system  shall  allow  movement  in  six  degrees  of 
freedom.  Movement  in  each  degree  of  freedom  shall  be  capable  of  occur- 
ring independently  without  simultaneous  motion  in  any  other  degree  of 
freedom.  The  motion  and  limits  of  movement  in  any  degree  of  freedom 
shall  not  be  restricted  by  the  positioning  of  the  trainer;  for  example,  it 
shall  be  possible  for  maximum  pitchup  to  occur  when  the  trainer  com- 
partment is  positioned  at  the  upper  limits  of  vertical  translation.  Also, 
roll  limits  and  performance  shall  be  unaffected  by  trainer  compartment 
position.  It  is  emphasized  that  the  limits  of  this  paragraph  represent 
performance  criteria  and  not  actual  design  limits.  With  respect  to  the 
trainer  flight  compartment  axes,  the  displacement  limits  of  motion  shall 
be  in  accordance  with  the  following  guidelines: 

Vertical  translation- - The  vertical  displacement  of  the  trainer 
flight  compartment  shall  be: 
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where 

Ym  “ Total  vertical  displacement  in  inches. 

w = Most  predominant  frequency  felt  by  the  aircraft  crew  at 
the  design  velocity,  altitude,  and  configuration  under  a 
1 cps  r ms  vertical  gust  environment,  in  cycles  per 
sec  ond. 

Unless  otherwise  specified  in  the  detail  specification,  the  vertical  dis- 
placement shall  be: 

0 - 24  inches  minimum 
0 - 30  inches  maximum. 

Lateral  translation- - The  lateral  translation  shall  be  as  deter- 
mined by  acceleration  requirements. 

Longitudinal  translation — The  longitudinal  translation  shall  be 
as  determined  by  acceleration  requirements. 

Roll  displacement-- The  roll  displacement  shall  be  determined 
as  a function  of  the  maximum  roll  acceleration  experienced  in  the  aircraft 
The  minimum  displacement  limits  shall  be: 

+ 15°  for  high  performance  aircraft. 

+ 9°  for  medium  performance  aircraft. 

(NOTE:  The  distinction  between  high  performance  and  low  performance 

aircraft  will  be  made  by  the  procuring  activity.) 

Pitch  displacement- - The  pitch  displacement  shall  be: 

For  high  performance  aircraft:  +14° 

-9° 

For  medium  performance  aircraft: 

+(  + 1°)  or  14°,  whichever  is  smaller 

-9°  where  asT  = ang^e  attack  for  aircraft  stall 

Yaw  displacement — Unless  otherwise  specified  in  the  detail 
specification,  yaw  displacement  shall  not  ordinarily  be  simulated.  It 
shall  be  considered  carefully  for  VSTOL  aircraft  and  a rationale  for  its 
inclusion  or  exclusion  submitted  to  the  procuring  activity. 
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Motion  system  accelerations.  The  motion  system  accelerations 
shall  not  adversely  affect  the  performance  or  durability  of  any  other  com- 
ponent of  the  trainer,  including  instruments  and  instrument  indications. 
The  maximum  accelerations  shall  be  determined  as  follows: 


Vertical  acceleration  — Vertical  acceleration  shall  be  +0 . 8g 
above  or  below  normal  lg . 

Lateral  acceleration — Lateral  acceleration  requirements  shall 
be  determined  by  the  lateral  acceleration  experienced  in  the  aircraft  under 
a 2 feet  per  second  rms  gust  condition,  or  as  specified  in  the  detail 
specification. 

Longitudinal  acceleration-- The  requirements  for  longitudinal 
acceleration  shall  be  determined  from  the  accelerations  experienced  in  the 
aircraft  when  thrust  minus  drag  (T  - D)  is  a maximum,  or  as  delineated 
in  the  detail  specification. 

Roll  acceleration — The  maximum  roll  acceleration  provided 
shall  be  consistent  with  operational  performance.  Unless  otherwise  speci- 
fied, the  lowest  acceptable  value  for  maximum  acceleration  shall  be  J-50° 
per  second^,  * 

IPitch  acceleration — The  pitch  acceleration  shall  be  as  deter- 
mined from  the  aircraft  pitch  acceleration,  consistent  with  operational  per- 
formance. Unless  otherwise  specified  in  the  detailed  specification,  the 
minimum  value  for  maximum  pitch  acceleration  shall  be  ^2  5°  per  second^. 


Yaw  acceleration — Yaw  acceleration  shall  be  in  accordance 
with  the  detail  specification. 

The  acceleration  provided  in  all  degrees  of  freedom  shall  be  realistically 
coordinated.  The  exact  degree  of  motion  simulation  to  be  provided  shall 
be  compared  to  comparable  aircraft  motions  and  presented  to  the  procur- 
ing activity  for  approval. 


Device  2B24: 


Motion  system  limits.  The  motion  system  shall  allow  movement 
in  four  degrees  of  freedom.  Movement  in  each  degree  of  freedom  shall  be 
capable  of  occurring  independently  without  simultaneous  motion  in  any  other 
degree  of  freedom.  The  motion  and  limits  of  movement  in  any  degree  of 
freedom  shall  not  be  restricted  by  the  positioning  of  the  trainer;  for  ex- 
ample, it  shall  be  possible  for  maximum  pitchup  to  occur  when  the  trainee 
station  is  positioned  at  the  upper  limits  of  vertical  translation.  Also,  roll 
limits  and  performance  shall  not  be  affected  by  trainee  station  position. 

It  is  emphasized  that  the  limits  of  this  paragraph  represent  performance 
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criteria  and  not  actual  design  limits.-  With  respect  to  the  trainee  station 
axes,  the  displacement  limits  of  motion  shall  be  in  accordance  with  the 
following  guidelines; 

(a)  Vertical  displacement  — The  vertical  displacement  of  the 
trainee  station  shall  be: 

0 - 24  inches  minimum 
0 - 30  inches  maximum 

(b)  Roll  displacement- - The  roll  displacement  shall  be  determined 
as  a function  of  the  maximum  roll  acceleration  experienced  in  the  aircraft. 
The  minimum  displacement  limits  shall  be:  _+15° 

(c)  Pitch  displacement — The  minimum  pitch  displacement  shall  be: 

+ 15° 

(d)  Yaw  displacement — The  minimum  yaw  displacement  shall  be: 

+ 150 

Degree  of  simulation.  With  respect  to  the  trainee  station  axes, 
the  physical  movement  of  the  trainee  station  shall  be  along  and  about  the 
Z-axis  (vertical  and  yaw),  about  the  Y-axis  (pitch),  and  about  the  X-axis 
(roll).  The  sensations  of  motion  shall  be  representative  of  sensations 
experienced  in  the  operational  aircraft  resulting  from  changes  in  attitude 
and/or  the  flight  path.  Representative  motion  caused  by  the  following  air- 
craft conditions  shall  be  provided:  Buffets,  blade  stall,  skids,  slips,  banks 

turns,  hovering  climbs,  dives,  rolls,  acceleration  and  deceleration,  re- 
lease of  stores,  vibrations,  oscillations,  touchdown  attitude  and  impact, 
and  control-induced  changes  in  the  exterior  configuration  of  the  aircraft. 

Motion  system  computations.  The  physical  motion  system  move- 
ment shall  be  determined  by  computations  based  upon  six  degrees  of  air- 
craft freedom.  Movement  along  or  about  all  axes  of  the  motion  system 
shall  be  correctly  correlated  with  the  motion  of  the  simulated  aircraft. 

All  aircraft  stability  derivatives  shall  be  accounted  for  in  such  a manner 
that  aircraft  movement  in  any  degree  of  freedom  shall  correctly  influence 
movement  along  or  about  every  axis  of  the  motion  system.  The  motion 
system  shall  respond  correctly  to  aircraft  center  of  gravity  or  center  of 
pressure  movements,  including:  fuel  depletion,  cargo  drops,  pod  drops, 

loss  of  tail  rotor  and/or  loss  of  tail  rotor  gear  box,  and  aerodynamic 
effects.  The  acceleration  of  the  trainee  station  in  any  degree  of  freedom 
shall  not  exceed  the  aircraft  acceleration  experienced  under  similar  flight 
and  configuration  conditions  of  the  operational  aircraft  simulated. 
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Operational  performance.  The  motion  system  design  shall  em- 
phasize onset  of  accelerations  to  provide  correct  physical  sensations  of 
motion  to  the  trainee.  Following  the  initial  transient  force,  the  cockpit 
shall  return  slowly  and  imperceptibly  to  normal  straight  and  level  flight 
attitude,  except  that  the  cockpit  shall  maintain  an  appropriate  amount  of 
the  commanded  pitch  attitude.  The  aircraft  instrument  indications  shall 
reflect  the  actual  simulated  flight  condition  regardless  of  cockpit  attitude. 
There  shall  be  no  noticeable  time  error  between,  instrument  response  and 
trainee  station  movement  except  as  appropriate  to  a given  instrument  such 
as  rate  of  climb. 


Motion  system  accelerations.  The  motion  system  accelerations 
shall  not  adversely  affect  the  performance  or  durability  of  any  other  com- 
ponent of  the  trainer,  including  instruments  and  instrument  indications. 
The  maximum  accelerations  shall  be  determined  as  follows: 


(a)  Vertical  acceleration- - The  lowest  acceptable  value  for  maxi- 
mum vertical  acceleration  shall  be  0 . 5g  above  and  below  the  normal  Ig 


(b)  Roll  acceleration-- The  lowest  acceptable  value  for  maximum 
roll  acceleration  shall  be  4-70°  per  second^ 

(c)  Pitch  acceleration- - The  lowest  acceptable  value  for  maximum 
pitch  acceleration  shall  be  -f-25°  per  second^ 

(d)  Yaw  acceleration  — The  lowest  acceptable  value  for  maximum 
yaw  acceleration  shall  be  +100°  per  second^ 

The  acceleration  provided  in  all  degrees  of  freedom  shall  be  realistically 
coordinated.  The  exact  degree  of  motion  simulation  to  be  provided  shall  be 
compared  to  comparable  aircraft  motions. 

Smoothness.  The  motion  system  shall  not  operate  erratically. 

The  system  shall  operate  smoothly  without  hunting  and  shall  not  snub 
against  cushion  stops  during  normal  operation.  The  actuator  system  shall 
smoothly  raise  the  trainer  in  a reasonable  level  attitude  to  the  initial  oper- 
ational position.  Oscillations  of  the  trainee  station  caused  by  motion  sys- 
tem instability  shall  not  occur.  Oscillations  of  the  trainee  station  or  any 
of  its  components  as  a result  of  structural*  hydraulic,  or  electrical  reso- 
nance shall  not  occur,  unless  such  movement  is  demanded  by  the  equations 
of  motion  of  the  flight  aircraft. 


Oscillations  and  vibrations.  Characteristic  continuous  and  peri- 
odic oscillations  and  vibrations  shall  be  qualitatively  simulated  by  the 
applications  of  frequencies,  amplitudes  and  directions  representative  of 
those  experienced  in  normal  and  emergency  flight  conditions  and 
maneuvers.  (End  of  exerpt  from  device  2f^4;v  motion  system  requirements.) 
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Some  design  practices  found  to  be  useful  by  various  simulator 
groups  as  well  as  selected  information  and  data  pertinent  as  guidelines 
for  design  are  set  forth  here. 


The  Link  proposal  for  the  Synthetic  Flight  Training  System, 

Device  2B24,  (Flexman,  et  al,  1968)  states  that  pitch,  roll  and  heave,  as 
well  as  yaw  (if  costs  can  be  justified)  motions  should  be  incorporated  into 
any  helicopter  trainer  motion  platform.  To  provide  the  yaw  cue,  yaw 
motion  can  be  resolved  into  an  angular  component  (rotation  around  a verti- 
cal axis  through  the  pilot1  s spinal  column)  and  translational  components 
(lateral  motion  plus  a negligible  amount  of  longitudinal  motion),  since  the 
pilot  is  about  6 feet  forward  of  the  center  of  gravity  for  the  UH-1D  heli- 
copter. Link  accepts  research  data  that  indicate  a detection  of  yaw  accel- 
eration at  about  1°  / sec^  with  a latency  of  2 seconds.  The  lateral  accelera- 
tion detectable  with  the  same  2 second  latency  is  approximately  0.02g,  or 
0.6  ft/sec^.  Since  each  l°/sec2  acceleration  about  a yaw  axis  six  feet  away 
produces  a lateral  acceleration  of  about  0. 1 ft/ sec'1,  it  appears  that  the 
rotary  component  will  reach  threshold  before  the  lateral  component  does  , 
and  hence  it  could  be  inferred  that  rotation  is  a more  significant  cue  for 
yaw  than  is  lateral  translation.  Similar  results  follow  the  examination  of 
thresholds  at  other  latencies;  at  1/2  second,  for  example,  their  values  are 
10°/sec^  for  angular  threshold,  0.3g  for  lateral  acceleration  threshold, 
and  a threshold  ratio,  at  the  6 ft.  distance,  of  1:  10,  rather  than  the  1:6 
obtained  at  2 seconds.  Thus,  a sufficient  yaw  cue  can  be  provided  by  rota- 
tion alone  around  a vertical  axis  near  the  pilot's  spinal  column,  without 
providing  the  lateral  translation  that  occurs  in  the  airciraft  when  it  yaws. 
(Note,  this  does  not  hold  for  the  CH-47A  helicopter  since  the  center  of 
gravity  is  19.5  feet  behind  the  pilot  and  the  threshold  ratios  for  the  air- 
craft are  on  the  order  of  1:2  to  1:3.  Hence,  the  desirability  of  lateral 
motion  for  the  CH-47A  cannot  be  ruled  out.)  With  respect  to  vertical 
motion  (heave),  the  same  design  philosophy  is  employed:  provide  a verti- 

cal travel  ample  to  allow  an  onset  cue  well  above  laboratory  threshold, 
and  a fading  to  the  neutral  position  with  subliminal  acceleration.  A 
straightforward  derivation  of  requirements  from  laboratory  thresholds 
is  not  possible,  due  to  the  absence  of  data  separating  the  effects  of  accel- 
eration from  that  of  jerk  (the  first  derivation  of  acceleration  with  respect 
to  time).  Experience  has  shown  that  a 24-inch  vertical  motion  (jKL2"), 
with  a velocity  limit  of  18"/ sec,  and  an  acceleration  capability  of  1 +_0.8g, 
is  satisfactory.  This  may  exceed  training  requirements.  A vibration 
capability  is  also  provided  to  represent  environmental  effects  and  the 
specific  cues  present  in  the  aircraft. 


Past  experience  with  simulation  fidelity  has  shown  that  motion 
systems  demonstrating  +12  inches  of  translational  motion  and  10  degrees 
of  angular  motion  have  fulfilled  many  needs  for  pilot  training  simulation 
(Clausen,  et  al,  1968). 
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A number  of  useful  guidelines  for  programing  the  motion  system 
has  been  forwarded  by  Borlace  (1967).  Among  these  are  the  following: 

. The  acceleration  used  should  be  that  of  the  cockpit 
and  not  that  of  the  center  of  gravity,  and  should 
account  for  aircraft  flexure. 

# Low  speed  responses  will  be  required  by  the  motion 
system  so  that  these  may  be  washed  out  by  accelera- 
tions below  the  threshold  for  the  vestibular  system, 
since  granularity  or  irregularity  is  immediately 
apparent  to  the  pilot.  This  necessitates  a wide 
dynamic  range  of  the  motion  system. 

. Pitching  the  cockpit  to  generate  the  effect  of  longitudinal 

ac c ele ration 3 while  it  gives  good  simulation  at  the  achieved 
angle,  yields  false  sensations  during  the  angular  motions 
(when  the  semi-circular  canal  signals  swamp  those  of  the 
otiliths  and  give  wrong  sensations).  The  same  applies 
to  rolling  the  cockpit  to  generate  sway  or  side  force, 

(however,  see  the  Wendt  and  Cohen  (1969)  calculation 
cited  earlier). 

• The  combination  of  pitch  and  simultaneous  heave  is 
unusually  effective  because  the  linear  acceleration 
output  of  the  otoliths  is  the  component  of  this  accel- 
eration resolved  along  its  plane  (not  the  vertical). 

Pitching  increases  this  component  and  so  pitch 
accentuates  the  corresponding  heave  motion  when  they 
are  applied  together. 

Apparently,  there  are  real  limitations  to  t he  number  of  degrees 
of  freedom  that  can  provide  realistic  motions  at  one  time  . These  will 
vary  with  the  particular  aircraft  and  displacements  required.  For  con- 
ventional aircraft,  pitch,  roll  and  normal  acceleration  should  be  adequate 
to  meet  most  requirements.  For  V/ STOL  aircraft,  yaw  should  be  included. 
When  aircraft  cockpits  are  far  removed  from  the  center  of  gravity,  side 
acceleration  should  be  included  possibly  in  place  of  yaw.  With  five  degree 
of  motion  simulation,  it  becomes  more  difficult  to  keep  the  simulator  free 
of  unwanted  motions  because  of  the  finite  restrictions  imposed  by  the  allow- 
able displacements  . Centrifugal  acceleration  is  imposed  by  rotation  and 
the  gimbal  adjustments  necessary  to  maintain  proper  cockpit  acceleration 
and  alignment.  ^ 


^Communication  with  John  Conant,  Melpar  Corporation,  Falls  Church, 
Virginia. 
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Boeing  (Graham,  1968)  has  specified  a simulator  (for  crew  station 
research  and  development)  where  pitch,  roll  and  heave  are  the  important 
axes.  Sway,  followed  by  yaw  are  important  when  the  pilot  is  displaced  some 
distance  from  the  center  of  gravity,  since  yaw  is  sensed  as  a sway  condi- 
tion. Yaw  becomes  increasingly  dominant  over  sway  when  the  cockpit  is 
close  to  the  center  of  gravity.  Surge  is  only  critical  for  the  simulation  of 
buffet  or  high-level,  long-term  longitudinal  acceleration.  Table  22  presents 
specifications  for  a six-degree  of  freedom  moving  base  simulator.  The 
simulator  drive  should  be  able  to  produce  vertical  vibration  frequencies 
from  0.  1 to  20.0  cps  at  amplitudes  sufficient  to  attain  an  rms  level  of  0 . 5g 
(at  frequencies  above  1.5  cps).  Lateral  frequencies  from  0.5  to  10.0  cps 
appear  necessary.  (The  lateral  rms  "j["  vibration  level  is  currently  un- 
determined.) Thus,  any  flight  simulator  having  a research  emphasis  should 
possess  at  least  four  degrees  of  freedom;  specifically  pitch,  roll,  heave 
and  sway,  and  be  constructed  such  that  the  remaining  two  degrees  can  be 
added  without  major  redesign. 

The  pivot  point  of  the  motion  platform  relative  to  the  location  of 
the  pilot  is  important  in  cue  presentation.  The  simulated  pivot  point  should 
approximate  the  actual  pivot  point  of  the  aircraft.  In  large  aircraft  where 
the  cockpit  is  well  ahead  of  the  yaw  pivot  point,  yaw  is  sensed  as  sway. 

When  the  pilot  is  located  near  the  pivot  point,  pitching  motion  is  experi- 
enced as  angular  acceleration  much  more  so  than  as  vertical  motion;  where 
the  cockpit  is  located  near  the  yaw  pivot  point,  the  rotational  element  of 
yaw  is  easily  perceived. 

Pilots  are  required  not  only  to  detect  movement  but  also  to  identify 
the  plane  of  the  movement,  hence  the  perception  of  motion  for  combined 
axes  is  of  some  importance  with  multi-degree  of  freedom  simulators. 

Very  little  useful  data  for  design  exist  on  this  issue.  Spatial  disorientation 
in  flight  can  occur  by  an  acceleration  vector  mistaken  for  the  gravitational 
vertical.  Head  movements  about  an  axis  other  than  the  axis  of  rotation 
(coriolis  effect)  can  modify  sensations  of  turning  (see  Stewart  and  Clark, 
1965).  Graham  (1968)  cites  the  assumption  that  the  vestibular  sensory 
system  can  be  used  to  create  the  illusion  of  movement,  but  how  to  exploit 
this  possibility  for  design  is  poorly  understood.  In  addition  to  possible 
vestibular  cross-coupling  effects,  interactions  presumably  take  place  be- 
tween the  vestibular  and  somesthetic  senses.  Further,  spatial  orientation 
is  normally  achieved  visually  and  visual  cues  are  dominant  over  somesthe- 
tic cues  in  the  competition  for  man's  attention.  This,  of  course,  provides 
the  rationale  for  giving  the  illusion  of  sustained  cockpit  motion  by  continu- 
ously driving  the  horizon  in  the  aircraft  attitude  indicator  following  only  a 
transient  movement  of  the  cockpit  itself. 

3.2.4.  1 Visual  Motion  Interactions.  The  use  of  visual  cues  to  augment 
motion  cues  is  of  considerable  interest  to  design  since  visual  motion  com- 
plements motion  sensed  proprioceptively . Combining  elementary  or 
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TABLE  22.  SIMULATOR  MOTION  SPECIFICATIONS 


Axis 

Displacement 

Acceleration* 

R&D  Use  Application 

Vertical 

+2  5 ft.  Minimum 

_+  8g  @ 1.0 

Turbulence,.  PIO, 

Required 

cps 

and  Terrain  Follow- 

+85 ft.  Desired 

"i 

ing 

Lateral 

+ 10  ft. 

32  ft/  sec^ 

'’Abnormal11  landing 
conditions  , turbu- 

Long itudinal 

+ 10  ft. 

32  ft/sec2 

) 

lence,  and  V/STOL 
operations 

Pitch 

+45° 

340  deg/ sec^ 

i 

Roll 

+60° 

680  deg/sec2 

\ All  of  the  above 

Y aw 

+45° 

400  deg / sec^ 

/ 

*"g"  levels  are  referenced  to  normal  gravity  as  the  zero  condition.  Hence, 

+ 1 vertical  "g"  doubles  the  aircrew's  weight,  while  -I  vertical  "g"  renders 
them  weightless  . 


(from  Graham,  1968.) 


r t 


NAVTRADEVCEN  69 -C -0298 - 1 


minimal  motion  such  as  a random  heaving  motion  with  a high  fidelity  visual 
velocity  cue,  has  given  rise  to  subjective  reports  from  pilots  that  they  are 
in  a moving  vehicle.  This  involves  signalling  the  pilot  of  the  onset  of  an 
acceleration  followed  by  motion  washout  but  continuing  with  the  high  fidelity 
visual  c ue  . 


The  predictive  value  of  an  acceleration  cue  in  a visual-motion 
cue  environment  is  well  understood.  Most  often,  an  acceleration  cue  leads 
the  visual  velocity  cue  in  time  of  perception.  Ivtatheny , Dougherty  and^ 

Willis  (1963)  indicate  that  in  systems  which  are  accelerated  at  about  7°/sec“, 
the  visual  and  motion  stimuli  are  responded  to  at  the  same  time.  However, 
when  the  acceleration  reaches  about  20°/  sec^  the  motion  cues  precede^the 
visual  cues  significantly.  At  an  angular  acceleration  of  about  25°/ sec  , 
motion  cues  precede  the  visual  cues  by  .4  to  .5  seconds. 

Providing  extra-cockpit  visual  cues  for  augmenting  motion  is 
highly  valued.  Visual  motion  can  effectively  mask  contradicting  physical 
motion  cues  resulting  from  washout.  However,  it  is  the  interaction  between 
visual  and  motion  cues  that  make  simulation  for  training  exceptionally  dif- 
ficult. When  visual  and  motion  cues  are  used  interactively,  both  must  be 
realistic  in  quality  and  temporal  relations.  This  is  much  easier  said  than 

done  . 


Motion  may  be  induced  visually  in  the  absence  of  physical 
motion,  but  the  use  of  vivid  extra-cockpit  motion  cues  (visually  induced 
motion  via  a projection  of  sky  and  terrain  onto  a screen)  in  the  absence  of 
physical  motion  has  limited  value.  The  absence  of  physical  motion  denies 
the  pilot  the  predictive  value  of  acceleration  cues  and  he  must  rely  on 
sensing  the  visual  velocity  (which  is  different  from  that  of  the  flight  envi- 
ronment). Also,  visual  attitude  cues  conflict  with  vestibular  cues  giving 
rise  to  undesirable  side  effects  in  the  simulator  such  as  nausea  and  vertigo. 
Studies  utilizing  multiple  cues  (visual  and  proprioceptive)  in  helicopter 
simulation  suggest  that  nausea  and  vertigo  result  from  distorted  optical 
motion  rather  than  from  a conflict  of  the  visual  and  proprioceptive  cues 
(Miller  & Goodson,  1958;  Sinacori,  1969). 


3 2.4.2  Vibration.  Vibration  is  also  an  issue  of  some  concern  to 

simulation.  It  is  an  important  design  feature  in  that  the  pilot  uses  vibra- 
tion, buzzing,  buffeting,  etc.  , as  a cue  for  how  the  aircraft  is  handling. 
Also,  certain  vibrations  of  the  aircraft  causes  the  pilot’®  he^d  and  eyes 
to  vibrate,  hence,  have  an  effect  on  visual  acuity  and  psychomotor  perfor- 
mance. Our  interest  in  vibration  centers  on  its  implications  for  the  design 
of  flight  simulators  in  terms  of  decisions  on  incorporating  vibration  into 
the  device  and  on  what  parameter  values  to  specify.  It  is  not  our  purpose 
to  provide  a review  of  the  data  on  vibration  effects  on  human  performance. 
A considerable  body  of  literature  exists  concerning:  the  physiological 
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effects  of  vibration  on  the  human  body;  the  effects  of  vibration  on  human 
performance  relative  to  visual,  psychomotor  and  cognitive  performance; 
and  the  conditions  arising  from  vibration  exposure,  such  as  motion  sick- 
ness and  fatigue.  A number  of  review  studies  have  been  published  detailing 
these  effects,  and  the  reader  is  urged  to  consult  these.  Noteworthy  among 
the  reports  are  the  surveys  by  Goldman  and  Von  Gierke  (I960),  Harris  and 
Crede  (1961),  Hornick  (1961),  Human  Factors  Journal  (October  1962), 
Townsend  (1967),  and  Roth  (1968).  Also,  a survey  of  data  on  comfort 
limitations  in  high  speed  ground  transports  has  been  published  by  Carstens 
and  Kresge  (1965). 


The  primary  vibration  spectrum  for  aircraft  that  is  most  criti- 
cal for  the  seated  pilot  falls  within  the  frequency  range  of  about  one -half 
to  twenty  cycles  per  second.  Frequencies  lower  than  this  do  not  have  a 
direct  effect  on  performance  but  may  induce  nausea;  feequencies  above  20 
cycles  per  second  can  be  effectively  isolated  from  the  pilot.  It  has  been 
determined,  for  example,  ( Graham , 196S/,  that  short-period  mode  pitch 

and  yaw  frequency  response  for  four  engine  jets  is  in  the  range  of  0.25  to 
1.0  cps;  bending  mode  response  produces  vibration  in  the  range  of  1 to  6 
cps;  vibrations  above  6 cps  are,  in  general,  produced  by  turbulence. 

Since  we  are  interested  in  the  effects  of  vibration  on  performance,  the 
level  of  acceleration  is  normally  less  than  one.  Within  the  1 to  20  cps 
range,  Hornick  (19  62)  suggests  that  the  following  kinds  of  activities  are 
affected  in  certain  frequency  ranges: 

Axis  F requency^  Effects 


Vertical  1-6  cps  Choice  reaction  time 


5-15  cps 
above  10  cps 
below  7.5  cps 
1 1 cps 


Tracking 
Visual  acuity 
Speech  intelligibility 
Breathing 


Transverse 

and 

Longitudinal  1-6  cps  Choice  reaction  time 

1.5  cps  Tracking 

1 . 5-5.5  cps  Breathing 


* Value  s given  are  frequencies  or  frequency  ranges  where 
effect  is  most  pronounced.  K*  i 
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The  perceptibility  of  vibration  (sinusoidal)  depends  upon  fre- 
quency and  amplitude.  In  general,  as  frequency  increases,  amplitude  can 
decrease  to  provide  the  same  level  of  perceptibility.  However,  for  the 
threshold  defined  as  ''definitely  perceptible,"  the  function  is  complex.  The 
fluctuations  in  amplitude  as  frequency  increases  to  produce  the  sensation  of 
definitely  perceptible  vibration  have  been  determined  by  Parks  (1962)  as 
follows: 


Frequency  (cps) 


Double  amplitude  (inches) 


1 

2 

5 

10 

15 

20 


25 


. 27 
. 17 
. 025 
.030 
. 035 
.016 
. 005 


The  conclusion  drawn  from  many  studies  is  that  neither  fre- 
quency nor  intensity  of  vibration  alone  determines  discomfort  or  perfor- 
mance degradation.  Neglecting  body  resonance  points  for  the  moment, 
discomfort  will  increase  as  frequency  increases  with  amplitude  held  con- 
stant; increasing  the  amplitude  at  a higher  frequency  increases  discomfort 
more  so  than  increasing  the  amplitude  at  a lower  frequency.  The  following 
parameters  must  be  considered  in  meaningful  specifications  of  vibration: 

. kind  of  vibration  (sine,  random,  pre-recorded 
or  programed) 


* frequency  (cycles  per  second) 


• displacement  (amplitude)  (inches  or  millimeters 
of  double  amplitude) 

» velocity  (inches/ sec,  millimeter s / sec) 


acceleration  (ft/sec^) 
jerk  (inches/ sec ^) 


Anecdotal  evidence  indicates  that  vibration  cues  distinctly  im- 
prove the  realism  in  the  control  of  the  simulator.  Vibration  is  especially 
important  in  helicopter  simulation.  Various  reports  of  helicopter  acci- 
dents due  to  confusion  and  spatial  disorientation  are  attributable  to  a mix 
of  degraded  visibility  conditions,  coriolis  effects,  vibration,  noise  and 
oscillation  from  the  vehicle's  inherent  instability . The  vibration  is  not 
simply  vertical  or  horizontal,  but  contains  motion  components  in  every 
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direction.  Additionally,  the  interaction  of  the  rotor  blades  with  portions 
of  the  fuselage  causes  vibration  containing  many  frequency  components. 
Magnitude  of  acceleration  and  amplitude  vary  similarly.  Helicopter  vibra- 
tion lies  predominantly  in  the  spectrum  of  10-70  cps  with  amplitudes  rang- 
ing from  0.4  mm  for  70  cps  to  2.4  mm  for  10  cps  (HornicK,  1961).  Because 
of  the  high  amplitudes  involved  for  low  frequencies,  concern  should  be 
shownfor  vibrations  below  20  cps  . 


Matheny  and  Wilkerson  (19  66)  cite  the  work  of  Kretz  with  the 
Giravions  Dorand  light  helicopter  simulator  for  training  French  pilots. 
Vibratory  motion  uncorrelated  with  changes  in  visual  displays  produced 
significant  savings  in  flight  training  time.  The  effect  produced  is  that  of 
breaking  the  simulator  loose  from  the  ground  and  thus  increasing  the  real- 
ism of  the  training  situation.  Although  not  tested  experimentally,  motion 
simulation  of  this  vibrating  type  may  enhance  transfer  of  training  at  a rela- 
tively low  cost. 

In  the  development  of  an  air-to-air  combat  simulator  ^ the  full 
vibration  spectrum  of  the  aircraft  is  deemed  essential  in  simulation.  ^Pilots 
rely  on  the  feel  of  vibration  to  judge  how  close  they  are  flying  to  the  limits 
of  the  aircraft  capability.  Fighter  aircraft  become  uncontrollable  at  the 
limits  of  capability  rather  than  at  the  point  of  structural  failure.  Since  the 
pilot  cannot  with  surety  rely  on  his  instruments  while  attending  to  a foe  to 
determine  how  close  he  is  to  the  limits,  he  must  rely  on  feel  and  sound  to 
determine  safety  range.  Thus,  simulation  must  accurately  represent  the 
sound  and  vibration  spectrum  throughout  all  speeds,  attitudes  and  altitudes 
of  the  aircraft.  Simulation  accuracy  is  particularly  important  in  the  region 
where  aircraft  control  is  lost.  Emphasis  must  also  be  placed  on  isolating 
the  vibration  from  the  visual  display  system  so  as  not  to  impair  visual 
acuity  except  as  the  man’s  head  vibrates.  Therefore,  vibration  is  best 
limited  to  the  seat  and  to  the  controls  normally  used  during  combat  man- 
euvers. Clousing  (1964)  has  reported  improved  realism  of  the  minimum 
unstick  speed  maneuver  on  a fixed  simulator  by  vibrating  a pneumatic  seat 
cushion  and  installing  a control  column  shaker. 


The  vibration  tolerance  curve  for  military  aircraft  has  been 
established  at  0.1  g (at  the  man)  between  1 and  20  cps  (Goldman,  1961); 
Wolbers  (19  59)  has  suggested  a limit  of  0.00015  inches  double  amplitude 
at  frequencies  below  60  cps.  Ho  consistent  limit  has  been  specified  for 
exposure  to  low  frequency  vibration. 


^"Communication  with  John  Conant,  IvLelpar  Corporation,  Falls  Church, 
Virginia. 
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Graham  (1968)  states  that  vibrat’on  reproduction  poses  no 
serious  design  problems  but  reproduction  of  large  displacements  below 
1 cps  is  a design  problem  having  no  simple  solution. 

3.2.5  Research  Issues.  A system  of  precise  quantitative  information  is 
not  available  as  a design  guide  for  motion  simulation.  No  organized  set  of 
"standards"  are  available  that  provide  guidance  on  the  design  of  motion 
systems  for  training  simulators.  To  be  sure,  there  exists  a body  of  knowl- 
edge relevant  to  simulator  design  (and  this  information  continues  to  grow) 
but  it  falls  considerably  short  of  the  data  requirements  for  human  factors 
design.  The  reasons  for  this  and  the  shortcomings  in  the  information  base 
have  already  been  discussed.  Because  of  this  state  of  affairs,  whatever 
success  motion  systems  have  enjoyed  in  flight  trainers  has  resulted  more 
from  an  empirical  approach  to  design  rather  than  from  well  documented 
design  principles  and  data. 

We  are  proposing  here  a number  of  human  factors  design  issues 
that  may  be  examined  profitably  to  enhance  the  information  base  on  wh ich 
design  decisions  are  predicated.  These  research  issues  are  outlined 
below.  They  are  not  ranked  in  any  order  of  primacy  nor  do  they  suggest 
similar  levels  of  effort. 

a.  An  information  base  on  sensory  thresholds  for  acceleration 
forces  within  the  envelopes  pertinent  to  training  device  design  is  needed. 
The  emphasis  here  is  on  task  situations  and  motion  parameters  relevant 
to  current  sophisticated  flight  simulators  and  other  complex  vehicle  types. 
The  basic  laboratory  data  on  sensory  thresholds  that  have  been  assembled 
during  this  century  are  not  directly  applicable  to  simulator  design  require- 
ments primarily  because  of  the  relatively  simple  task  situations  investi- 
gated, the  non-comparability  among  studies  (as  to  latency  times,  task 
differences)  and  the  concentration  on  single  axes  of  motion.  Also,  labora- 
tory studies  characteristically  require  the  subject  to  attend  to  a single  task 
requirement  (unlike  the  much  higher  operational  task  loadings)  in  which 

he  is  alerted  to  the  most  minimal  changes  in  motion  cues.  At  best,  labora- 
tory data  indicate  what  the  man  cannot  perceive  or  utilize  operationally. 
Thus,  the  effective  thresholds  for  design  will  be  raised  considerably. 

What  the  pertinent  relationships  are  should  be  determined  systematically. 

b.  Allied  to  (a)  above,  is  the  need  for  a data  base  on  the  percep- 
tion of  motion  (thresholds  and  definition,  direction  and  quality  of  the  ex- 
perience) for  combined  axes.  Of  concern  are  the  multi-axis  interactions 
associated  with  current  sophisticated  flight  simulators.  It  is  well  under- 
stood that  the  effective  axis  of  acceleration  in  the  operational  environment 
seldom  falls  on  any  one  orthogonal  axis  but  is  a vector  with  components  of 
each. 
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c«  In  view  of  the  difficulties  in  achieving  full  representation  of 
physical  motion  in  the  simulator,  a crucial  research  issue  concerns  an 
examination  of  cue  values  in  less  than  the  complete  motion  axes  simulation. 
Since  representing  6 degrees  of  freedom  cannot  be  achieved  except  for  brief 
time  durations,  the  question  of  importance  is  what  can  be  dropped  out  with- 
out seriously  compromising  the  training  capability?  This  is  to  say,  how 
little  motion  can  you  get  away  with  vis-a-vis  specific  task  requirements? 

The  issue  revolves  around  the  priority  of  motion  axes  and  how  many  para- 
meters to  provide  for  defined  situations.  An  immediate  question  to  resolve 
is  whether  6 degrees  of  freedom  is  more  desirable  than  5 or  4 in  light  of 
the  ability  to  install  them  satisfactorily?  Operating  practices  have  most 
often  settled  for  four  or  less  axes.  There  is  the  strong  suggestion  in  the 
literature  that  fewer  motion  axes  (e.g.  , two)  augmented  by  other  motion 
cues  will  yield  quite  satisfactory  results  for  purposes  of  training.  For 
example  , the  employment  of  cockpit  tilting  to  augment  the  physical  motion 
which  is  introduced  and  then  faded  back  after  a short  time  duration  may 
have  useful  cue  value.  The  same  may  be  said  for  applying  pressures  on 
critical  body  points  as  motion  cues.  In  total,  the  requirement  is  for  a 
programatic  study  of  the  available  motion  parameters  and  values  for  repre- 
sentative aircraft  tasks.  The  study  should  be  accomplished  with  operational 
simulation  equipment. 

d.  Allied  to  (c)  above,  is  the  need  for  systematic  research  on 
visuail  and  motion  interactions  in  flight  simulation.  There  are  ample  sug- 
gestions from  operating  practice  that  when  a high  fidelity  visual  display 
is  coupled  with  rudimentary  or  even  imprecise  physical  motion,  the  pilot 
feels  he  is  in  a moving  vehicle.  For  example,  anecdotal  evidence  indicated 
that  a random  heaving  motion  coupled  with  high  fidelity  displays  will  achieve 
substantial  task  fidelity.  The  goal  is  to  put  into  perspective  the  options 
available  in  backing  off  from  total  motion  simulation  to  achieve  training 
objectives.  Similarly,  data  are  needed  which  test  the  validity  of  less 
than  precise  fidelity  in  motion  representation,  i.e.  , what  are  the  cue 
value  relationships? 

Related  to  this  area  is  the  effect  of  optical  distortion  in  per- 
formance. Visual  (apparent)  motion  that  is  distorted  (i.e.  , inexact 
simulation  of  the  visual  cues  which  occur  in  operational  flight)  has  been 
cited  as  the  cause  of  nausea  and  dizziness  in  the  device  2-FH-2  Hover 
Trainer  and  not  due  to  the  conflict  between  visual  and  motion  cues  (Miller 
and  Goodson,  1958)  . Gibson  (Horowitz,  1964)  states  that  the  geometry  of 
motion  perspective  is  not  well  worked  out.  It  may  be  of  some  value  to 
develop  principles  for  partialing  out  optical  distortions  in  the  coupling  of 
visual  and  motion  dynamics  in  flight  simulators. 

e „ The  issue  of  the  scaling  down  of  motion  forces  in  simulator 
design  is  of  some  concern.  Partial  force  simulation  involving  a reduction 
in  the  dynamic  range  required  of  the  motion  platform  appears  to  be  a 
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desirable  design  option,  particularly  in  a bigb  fidelity  visual  display  and 
platform  motion  coupling  context.  Basic  data  and  design  principles,  how- 
ever, are  lacking,  and  research  is  needed  to  place  the  design  relationships 
into  perspective.  Of  some  interest  is  the  validity  of  objective  performance 
measures  in  the  scaled  down  motion  cue  environment  and  the  transfer  of 
training  effects  of  this  simulated  environment  to  the  actual  aircraft. 

f.  A desirable  effort  is  a study  which  organizes  and  collates,  in 
guideline  form,  the  design  procedures  found  to  be  currently  most  success- 
ful in  motion  platform  construction  as  practiced  by  simulator  manufacturers 
This  should  be  undertaken  for  a variety  of  task  requirements  for  both  train- 
ing and  research  simulators  and  with  the  recognition  of  the  proprietary 
interests  involved. 

g.  Enough  evidence  has  been  assembled  that  indicates  that  motion 

can  be  induced  by  techniques  other  than  moving  the  entire  cockpit  on  a 
platform.  Since  man  has  difficulty  in  distinguishing  between  vestibular 
and  somesthetic  sensations,  subjectively  valid  motion  cues  may  usefully 
augment  the  platform  motion.  Auxiliary  devices,  such  as:  1)  computer 

controlled  pneumatic  seat  cushions  and  restraining  devices  such  as  har- 
nesses and  seat  belts,  and  2)  the  use  of  shearing  forces,  i.e.  , stimulation 
of  touch  and  pressure  points  on  the  skin,  may  provide  additional  effects  of 
sustained  acceleration  meaningful  to  the  man.  Quantitative  relationships 
should  be  developed  for  the  employment  of  these  techniques  in  sophisticated 
training  devices.  Again,  operational  simulation  equipment  should  be  uti- 
lized in  these  investigations. 

h.  Basic  data  are  needed  on  vibration  thresholds  over  the  perti- 
nent range  of  frequencies  and  task  and  maneuver  requirements.  These 
data  should  be  obtained  on  operational  simulation  equipment.  In  addition, 
studies  should  be  undertaken  to  determine  the  effects  of  random  uncor- 
related vibrating  motion  on  performance.  It  is  also  reasonable  to  deter- 
mine the  transfer  of  training  of  uncorrelated  motion  vs.  correlated  motion 
to  the  actual  aircraft. 


i.  A review  study  is  desirable,  tailored  fox-  the  human  factors 
specialist  responsible  for  training  device  design,  which  places  into  per- 
spective issues  of  spatial  disorientation,  illusions  of  movement  and  nausea 
that  are  attributable  to  simulator  design  but  not  correlated  with  events  in 
the  operational  environment.  Guidelines  should  be  organized  to  call  atten- 
tion to,  and  to  develop  means  for  minimizing  these  artifacts  in  design. 

j.  It  is  noted  as  a general  observation  that,  a proportionally  large 
number  of  motion  studies  have  been  concerned  with  comparisons  of  motion 
with  no  motion  on  performance  in  the  simulator  . Obviously , it  would  be 
desirable  to  acquire  a transfer  of  training  data  base  using  the  motion-no 
motion  paradigm  with  operational  simulation  equipment  and  the  operational 
aircraft .. 
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3.3  VEHICLE  CONTROL  REQUIREMENTS1 

This  chapter  considers  human  factors  design  requirements  for  simu- 
lating aircraft  vehicle  control.  The  basic  issues  concern  the  design  options 
available  for  achieving  transfer  of  training  and  center  on  the  relationships  of 
simulation  conditions  to  training  effectiveness.  Two  quite  opposite  view- 
points are  currently  held.  One  approach  assumes  that  the  desired  design 
is  one  wherein  the  simulated  controls  are  made  to  match  vehicle  controls 
as  close  as  possible  (i.e.  , the  best  training  simulators  bear  the  closest 
physical  resemblance  to  the  aircraft).  The  capabilities  and  the  relatively 
reasonable  costs  of  current  high  speed  digital  computers  support  this  con- 
cept of  complex  simulation.  Algebraic  operations  are  easily  accomplished 
in  the  digital  computer,  hence  there  is  a tendency  to  include  a great  many 
terms  in  the  relevant  equations,  even  though  the  effects  of  many  of  them 
may  be  unknown  or  quite  negligible  for  training  purposes. 

The  opposite  approach  suggests  that  a reduction  in  physical  fidelity  of 
simulation  (e.g.  , omission  of  terms  in  the  equations)  can  be  accomplished 
without  significantly  affecting  trainee  performance.  Systematically  reduc- 
ing the  level  of  aerodynamics  simulation  while  not  compromising  perfor- 
mance, introduces  significant  economies  in  computer  hardware  and  associ- 
ated software  programing.  Additionally,  it  is  desirable  to  depart  from 
fidelity  of  simulation  to  provide  additional  cues  supportive  of  training  but 
not  found  in  the  operational  system.  Assuming  cost  minimization  as  a 
goal,  design  and  fabrication  costs  may  be  reduced  by  subtracting  unnee de d 
simulator  fidelity  from  a contemplated  design,  and  utilization  costs  may  be 
reduced  by  adding  extra  cues  to  the  simulator  which  will  enhance  the  trans- 
fer of  training  or  reduce  the  training  time. 

Although  current  and  future  computation  capabilities  encourage  design 
of  simulated  controls  which  duplicates  the  characteristics  of  the  operational 
aircraft,  the  discussion  in  this  chapter  centers  on  the  alternative  approach, 
that  of  backing  off  from  fidelity  without  compromising  training  effectiveness 
The  material  presented  concerns  the  following  topics: 

• the  purpose  of  simulator  controls  (as  distinct  from 
aircraft  controls) 

m desired  control  fidelity 

m evaluative  criteria  for  simulator  controls 

# research  issues. 

1This  chapter  was  prepared  by  Dr.  Herbert  Smithline  of  Dunlap  and 
Associates  , Inc  . 
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3.  3.  1 Purpose  of  Simulator  Controls,  Though  they  may  appear  to  have  the 
same  purpose,  the  controls  of  an  aircraft  serve  a fundamentally  different  pur- 
pose than  the  controls  of  a simulator.  Very  simply,  the  controls  of  an  air- 
craft are  designed  to  guide  the  aircraft;  but  the  controls  of  a simulator  are 
designed  to  train  the  student.  Thus,  the  designer  need  not  hesitate  to  deviate 
from  physical  fidelity  if  by  so  doing,  equivalent  training  is  accomplished  at 
lower  costs. 


For  training  purposes,  the  student  is  taught  to  do  with  the  simulator 
controls  what  the  skilled  pilot  does  with  the  aircraft  controls.  Aircraft  con- 
trols serve  a dual  purpose.  They  are  used  to  alter  the  state  of  an  aircraft,  or 
they  are  used  to  maintain  the  status  of  an  aircraft  in  spite  of  disturbing  forces. 
McDonnell  (1968)  states  that,  control  implies  the  imposition  of  commands  on 
the  system  and  also  the  suppression  of  disturbances.  The  inputs  of  the  pilot 
to  the  aircraft  may  be  pure  commands  (which  are  functions  of  time  alone)  or 
may  depend  on  some  vehicle  deviation  from  a desired  state  of  motion;  so 
command  operations  are  open-  and  closed -loop  in  nature.  Although  the  open- 
loop  characteristics  can  have  a large  influence  on  pilot  workload,  ratings  (of 
handling  qualities)  tend  to  depend  on  closed-loop  control  characteristics  be- 
cause most  deficiencies  appear  only  under  these  conditions. 


3.  3.  1.  1 Kinds  of  Tasks.  The  pilot  trainee  is  called  upon  to  engage  in  con- 
tinuous interactive  tasks  as  well  as  non -inter active  sequential  procedures. 

The  training  simulator  controls  should  assist  the  pilot  to  perform  successively 
more  complex  closed-loop  tasks,  as  well  as  to  progressively  improve  his  se- 
lection and  interleafing  of  required  open-loop  tasks.  Design  should  be  more 
concerned  with  dynamic  control  response  for  closed-loop  tasks  ; while  control 
location  and  configuration  should  be  the  concern  for  the  open-loop  variety. 

This  is  not  to  suggest  that  the  demarcation  between  open-  and  closed-loop  be- 
havior is  always  clear-cut.  Skillful  behavior  is  essentially  closed -loop  initially. 
However,  as  skill  increases,  behavior  takes  on  the  characteristics  of  open-loop 
behavior,  as  there  is  progressively  less  need  for  feedback.  That  is,  the  check- 
ing and  correcting  feedback  is  required  only  on  an  intermittent  basis,  and 
simple  skillful  acts  may  be  chained  without  conscious  feedback  for  each  simple 


act. 


3.  3.  2 Control  Fidelity.  A popular  belief  is  that  the  best  simulator  is  the 
one  which  most  perfectly  matches  the  simulated  aircraft,  hence  it  would  be 
assumed  that  simulator  controls  should  match  vehicle  controls  as  closely  as 
possible.  However,  perfect  physical  fidelity  does  not  necessarily  enhance 
training  value,  particularly  in  the  earlier  stages  of  training.  Optimum  trans- 
fer of  training  is  promoted  by  devices  and  situations  which  provide:  guidance 

for  desirable  behavior,  knowledge  of  the  results  of  performance,  reinforce- 
ment of  proper  behavior,  motivation  for  the  trainee,  and  also  encourage  the 
development  of  a learning  set.  Obviously,  it  is  desirable  in  some  cases  to 
depart  from  aircraft  fidelity  to  provide  special  cues  and  other  supports  to 
achieve  training  goals.  In  addition,  it  may  be  desirable  to  retreat  from  full 
aircraft  fidelity  to  realize  savings  in  simulator  design  and  fabrication  costs, 

O rticularly  if  the  additional  costs  are  incurred  with  no  practical  increase  in 
, rceptual  equivalence.  21 8 
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3.3.2. 1 Fidelity  for  Purposes  of  Training.  As  noted  above , fidelity  of 

simulator  controls  to  aircraft  controls  is  not  a primary  goal;  whereas  ade- 
quate transfer  of  training  is,  thus,  it  is  often  quite  desirable  to  depart  from 
physical  fidelity  to  achieve  the  desired  transfer  effects.  Since  training  is 
facilitated  if  easier  tasks  lead  to  difficult  ones,  and  the  ease  or  difficulty 
of  a task  may  be  influenced  by  simulator  control  characteristics,  the  follow- 
ing kinds  of  deviations  from  realism  are  beneficial  to  the  training  process: 

• quickening  the  response  to  control  motion  so  that 
a trainee  can  more  readily  see  errors  and  correct 
them 

. adding  special  cues  to  the  "feel11  of  the  controls 
(for  example , artificial,  i.e.  , nonrealistic  , 
control  vibration  may  be  used  to  shape  trainee 
responses) 

m adjusting  the  control  gain,  to  make  the  simulator 
fly  more  easily  than  the  aircraft  it  represents. 


In  fact,  control  characteristics  may  be  automatically  modified  as  a function 
of  trainee  performance  level  as  a means  of  achieving  adaptive  training.  In 
adaptive  simulation,  the  task  can  be  manipulated  by  modifying  the  external 
forcing  function.  Here  it  is  suggested  that  the  task  may  be  altered  by 
modifying  the  control  dynamics,  e.g.  , by  shortening  system  response  time. 


In  addition  to  adding  artificial  cues  to  the  operation  of  the  con- 
trols to  facilitate  tra.nsfer  of  training,  control  cues  may  be  augmented  so 
that  a novice  could  be  taught  to  perceive  what  a veteran  recognizes  as 
significant  in  the  feel  of  the  controls. 


3. 3.2, 2 Motivational  Benefits  of  Realistic  Controls.  Good  perceptual 
fidelity  of  the  controls  adds  a great  deal  of  face  validity  to  the  training 
situation,  especially  for  the  advanced  student  who  has  had  substantial 
exposure  to  the  aircraft  being  simulated.  As  long  as  a trainee  can  say, 

. .all  exercises  up  to  date  have  felt  more  like  playing  darts  than  flying 
an  aeroplane.  . . 11  (Craik,  1966),  he  will  not  be  properly  motivated  for  the 
training  experience.  In  evaluating  pilot  ratings  of  handling  qualities  , 
McDonnell  (1968)  found  that  the  ratings  were  primarily  based  on  1)  simula- 
tor performance,  2)  required  pilot  equalization,  and  vehicle  stick  charac - 
teristics.  Orlansky  (1948)  reported  that  experienced  fighter  pilots  rated  x 
horizon  reference  and  stick  feel  as  the  primary  sources  of  pues  during 
many  maneuvers.  A stick  with  feel  is  equivalent  to  a host  of  flight  instru- 
ments . Accordingly , it  is  not  surprising  that  if  the  controls  do  not  feel 
right  to  an  experienced  pilot,  his  confidence  in  the  simulator  will  be  under- 
mined to  such  an  extent  that  his  training  will  be  hindered.  It  behooves  the 
designer  therefore,  to  provide  simulator  controls  that  can  be  set  to  feel 
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like  those  of  the  aircraft.  At  times  in  the  training  cycle,  particularly 
early  in  the  cycle,  it  may  be  desirable  to  make  the  simulator  controls 
Measy, 3 * * * * * * *  11  or  incorporate  special  cues  but  at  some  point  the  trainee  has  to 
be  motivated  by  the  knowledge  that  he  is  experiencing  a faithful  representa- 
tion of  the  aircraft.  In  a subsequent  section,  we  shall  discuss  the  important 
factors  for  perceptual  (not  physical)  fidelity.  First,  however,  factors  other 
than  !lfeelM  that  can  affect  the  usefulness  of  simulator  controls  are  discussed. 

3.  3. 2. 3 Fidelity  of  Configuration.  In  addition  to  improper  "feel,  11 
trainees  tend  to  be  disturbed  by  controls  which  are  not  configured  correctly 
(e.g.  , because  the  simulator  represents  a model  of  the  aircraft  older  than 
the  one  with  which  pilot  is  familiar).  As  a practical  matter,  to  maximize 
positive  transfer,  design  should  consider  the  use  of  modularized  or  other- 
wise easily  re  configurable  control  elements.  Improperly  configured  con- 
trols can  have  a negative  effect  on  trainee  attitude  . r 

3. 3. 2.4  Fidelity  Requirements  for  Interactive  Tasks.  Much  of  a trainees 
time  in  a flight  simulator  is  spent  on  closed-loop  tasks.  He  learns  to  main- 
tain the  condition  of  the  simulated  aircraft  despite  disturbing  forces  , or  he 
learns  how  to  alter  the  condition  from  state-to - state  in  a properly  controlled 
manner.  Current  simulators  are  capable  of  a wide  range  of  control  char- 
acteristics to  support  the  training  procedure.  Circuits  can  produce 
accurate  overall  force  gradients  and  also  represent  such  mechanical  fea- 
tures as  hysteresis,  dead-band,  and  stick  vibration.  Simulator  designers 
have  become  increasingly  able  to  duplicate  the  physical  world  in  a simula- 
tor, though  at  increasing  costs.  The  question  naturally  arises,  can  some 
physical  fidelity  be  dispensed  with,  without  sacrificing  perceptual  equiva- 
lence. For  example,  Demaree,  Norman  and  Matheny  (1965)  state  that  the 
rising  cost  of  providing  high  simulation,  fidelity  makes  it  imperative  that 
studies  be  conducted  in  an  effort  to  determine  acceptable  levels  of  reduction 
in  fidelity  with  accompanying  reductions  in  simulation  costs.  In  the  follow- 
ing paragraphs  , we  shall  discuss  closed-loop  task  fidelity  as  it  is  affected 
by  the  aerodynamic  equations,  the  calculation  of  equation  parameters,  and 
the  program  cycle  time. 

3.  3.2.4.  1 Simplification  of  Aerodynamic  Equations  . Studies  have 

attempted  to  ascertain  perceptually  significant  characteristics  of  a simula- 

tor . A representative  example  is  the  work  of  Wilkerson,  Norman,  Matheny, 

Demaree  , and  Lowes  (1965)  , which,  demonstrated  that  the  aerodynamic 

equations  of  the  simulated  aircraft  (and.  hence  the  motions  and  forces  on 
the  controls)  may  be  simplified  for  training  purposes.  They  set  to. zero 
such  terms  as  the  thrust  contribution  to  pitching  moment,  product  of. inertia 
contribution  to  yaw  rate  , etc  . The  results  favoring  the  use  of  simplified 

aerodynamic  equations  are  qualified,  however , with  the  recommendation 
that  the  findings  should  be  confirmed  on  amoving  base  simulator  , involv- 

ing realistic  flight  segments  (e.g.  , landing  approach  maneuvers)  with 

turbulence. 
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Given  the  aerodynamic  equations,  Wilkerson,  et  al  (1965)  and 
Ellis,  et  al  (1968),  found  that  reduced  simulation  conditions  resulting 
from  use  of  rigid  airframe  aerodynamic  coefficients  may  well  be  adequate 
for  training.  However,  straight  line,  least  square  approximations  to  the 
flexible  coefficients  were  found  to  be  not  feasible.  For  convenience  of  the 
reader,  summaries  of  their  findings  are  shown  below.  Table  2 3 shows  the 
development  of  complete  and  incomplete  aerodynamic  equations.  Tables 
24  and  2 5 indicate  the  kinds  of  errors  in  longitudinal  and  lateral  parameters 
which  may  be  tolerated  (column  headed  "Rigid")  and  which  may  not  be 
tolerated  (column  headed  "Least  Squares"),  so  far  as  their  effect  on  trans- 
fer of  training  is  concerned. 

3. 3. 2. 4. 2 Increasing  Program  Cycle  Time.  In  addition  to  modifying 

the  aerodynamic  equations  used,  fidelity  may  be  reduced  and  simulator  cost 
decreased  by  increasing  the  program  cycle  time.  Program  cycle  time 
refers  to  the  repetition  interval  for  the  exchange  of  data  between  the  com- 
puter and  the  (simulated)  cockpit,  and  for  the  solution  of  the  simulation 
equations.  (Wilkerson,  et  al,  1965)  The  shorter  the  cycle  time,  the  f 
greater  the  fidelity  possible  in  a simulator.  Cycle  times  used  covered 
the  range  of  33.3  to  83.3  msec.  Hunt  (1967)  discusses  the  short  cycle 
time  for  a particularly  difficult  control  simulation.  "An  example  of  an 
unusual  control  force  domain  is  the  simulation  of  the  force  characteristics 
of  the  control  yoke  fore  and  aft  motion  on  an  unr.oot  ed  spring  tab  elevator 
system  when  the  aircraft  is  on  the  ground  and  the  tab  has  no  aerodynamic 
effect.  In  this  example,  the  problem  arises  from  the  complicated  sequence 
of  mechanical  linkage  pickup  and  snubbing  actions  occurring  throughout  the 
total  travel  of  the  control  yoke  with  attendant  abrupt  inertia  changes  reflec- 
ted to  the  control  column.  In  attempting  to  provide  realistic  simulation  of 
control  forces  throughout  all  foreseeable  operating  conditions,  the  designer 
of  a control  force  system,  must  provide  a servo  with  unusual  dynamic  char- 
acteristics. The  closed— loop  response  of  such  a system  is  controlled  by? 
computer-generated  parameters  , is  exceedingly  nonlinear  , and  involves 
closed- loop  frequencies  as  high  as  thirty  cycles  per  second.  " (p.  44.) 

More  reasonable  cycle  times  of  50  msec,  were  set  for  the  Universal 
Digital  Operational  Flight  Trainer  Tool  (UDOFTT)  developed  by  the  U.S. 
Naval  Training  Device  Center  and  the  U.S.  Air  Force.  However, 
Wilkerson,  Norman,  Matheny , Demaree  and  Lowes  (1965),  found  that  the 
program  cycle  time  for  the  UDOFTT  could  be  increased  to  83.3  msec,  for 
many  training  purposes,  using  the  O33  Gurk  integration  formula. 

Their  work  indicated  that  other  integration  formulas  might  permit  further 
lengthening  of  program  cycle  time  though  they  did  not  suggest  an  upper 
bound  to  cycle  time.  As  a limit,  we  might  expect  that  the  cycle  time  could 
probably  not  be  increased  beyond  125  ms. 

3.3. 2. 4. 3 Control  Type s.  The  previous  discussion  indicated  that  aero- 

dynamic equations  and  their  implementation  affect  feedback  to  the  trainee 
directly  through  his  controls  as  well  as  through  other  avenues  (e.g.  , 
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TABLE  23.  COMPLETE  AND  INCOMPLETE  AERODYNAMIC  EQUATIONS1 
Accs  le ration  Along  X - V7ind  Axis 


Conplete  Equation: 

A*  = Jk  [ - qs  {f2  CM)  + Cfs  CM)  + f6  CM)  f7  CCL)]  0^2}  + Tn  - WG  sin  rj  * 

No  Change 

Acceleration  Along  Y - Wind  Axis 

C Aerodynamic  and  Thrust  Compcavints  Only) 

Complete  Equation:  K^_  = K2  = ^14  =1 

Aya  = 1 J’qs  {_  fg  (M)  B + Ki  f10  CM)  6r  - K14  f u CM)  «aT0I  > - K2  Tn  cos  a sin  b]  . 

Inconplete  Equation:  Ki  = K2  = K3.4  = 0 

Aya  = i . qs  C-  fg  CM)  3 B. 

: CcntrLbutibn  to  side  force  due  to  rudder  deflection 

Kj  : Thrust  component  term 

K14  : Contribution  to  side  force  due  to  aileron  deflection 

■Hrhe  corcrolete  equations  were  developed  by  the  manufacturer  of  the  airplane 
which  was  simulated  and  the  incomplete  equations  were  developed  by 
Castellano  CTn  Press)  . y . r v ."y  y y.  y y '■ 
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TABLE  23.  COMPLETE  AND  INCOMPLETE  AERODYNAMIC  EQUATIONS  1 
(continued) 


Acceleration  Along  Z - Wind  Ads 

(Aerodynamic  and  Thrust  Components  Only) 


Complete  Equation: 


_ 1 


a za  = nr  qs  cl  ” tn  sin  “3. 


No  Change 


Roll  Acceleration  About  the  X - Body  A>ds 


Complete  equation:  K3  = K4  = K5  = l<qq  = 1 

• _ cos  ci  - rlzs  sin 

p = ; — 


Ixx 


J- 


+ K4  (IW^~  Tz z>  qi  r + k5  T— ct  p q 


*z 


+ c 


Ixx  . " 

Hzs  cos  g + rixs  sin  cx 

Izz 

2 -1 


'•1 


(Ixx  - lyy) 


J-zz 


PQl  “ 


£> 

Izz 


Ku  qx  ri  K3  pz.  , 


XX 


{1  - -<3 


- J*z 


Ixx  Izz 


}. 


Incomplete  Equation:  X3  = K»<.  = K5  = Krv  = 0 

**x<g  cos  ct  " «'^z<=;  sin  a ' 


P = 


Lxx 


K3  : Product  of  inertia  contrlbxrticn  to  rrll  rate , 
: Moment  of  inertia  contribution  to  roll  rate. 

2 Product  of  inertia  contribution  to  roll  rate . 
: Product  of  inertia  contribution  to  rate. 
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TABLE  23.  COMPLETE  AND  INCOMPLETE  AERODYNAMIC  EQUATIONS 
( continued) 


Rolling  Mcment  About  the  X Stability  Axis 


Complete  Equation:  (Kg  = 1) 

Mxs  = qsb  (C-f19  (M)  - f20  CM)  f2i  (h)3  B 

+ C-f23  (cL>  - f^3  (cL)  - f24  (cL)  f25  (M)3  Ci-f35  (M)  f36  003  saT0T 
+ Cf26  (M)  -.00052  CL]  tl-f37  (M)  f38  003 

+ C-f27  (W)+^29  CCl)  f30  (M)  - f28  (Cl)  3 Cl-f3g  (M)  fs8  003 

(p  cos  a + r sin  a) 


+ Kg  Cf32  (Cl)  3 Cl-f39  (M)  fse  00 


(r  cos  a - p sin  a)}. 


Incomplete  Equation  (Kg  = 0) 

K*s  = 9sb  {C~f19  CM)  - f20  (H)  f2i  00  3 B 

+ C-f23  (CL)  - <CL)  - f24  (CL)  f25  (M)3  Cl-f35  CM)  f3g  003  «aTOT 
+ Cf26  (M)  -.00052  Cl3  Cl-f37  (M)  f33  003  «r 

+ C-f27  (M)  + f2g  (CL)  f30  (M)  - f28  <CL)3  Cl-f39  CM)  f58  Ch)3  2C1XT7T1-1 
; (p  cos  «»  + r sin  a)), 

Kg  : Contribution  to  roll  moment  due  to  turydng  rate. 

Yaw  Acceleration  About  Z - Body  Axis 


Complete  Equation:  - 1 

£>  = Mzc;  COS  a + Hxs  sin  a + (1^  - Iyy)  + Jxz  p 

IzZ  11  Izz 


Lzz 

V Jxz  V, 

** K11  T5z  qlr 


Lzz 


Incomplete  Equation : Kji  - 0 


_ Mzs  cos  a + M*s  sin  » + (Ixx  — 


n 


r = 


, . Jv„ 
P9l  + 

lzz 


J-ZZ  Azz 

-11  : Product  of  inertia  contribution  to  yaw  rate. 
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TABLE  23.  COMPLETE  AND  INCOMPLETE  AERODYNAMIC  EQUATIONS1 
(continued) 
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TABLE  23.  COMPLETE  AND  INCOMPLETE  AERODYNAMIC  EQUATIONS1 
(continued.) 

Pitching  Moment  About  the  Y - Stability  Axis 
Complete  Equation : Kg  = Kg  = 1 

Mys  = qsc  { f40  (M)  + [-  f46  CM)  - f47  Ch)  f48  CM)  + 

+ C-  f51  <M)  - f52  CM)  f53  ^)]  zaiOTTT  «U 

- Cf54  CM)  + fss  Ch)  f56  CM) 3 6±s 

+ K8  CfS7  CM)  - f49  Ch)  f50  CM)]  2alfyrM 5^-3-  > 

- a 0833  Kg  Tjsj. 

Incoirplete  Equation:  Kg  = Kg  = 0 

. n,s  = qsc  (f40  CM)  + [-  f4S  CM)  - f47  Ch)  f48  CM)  + -Jjgii  ] Clj.  . 

+ C-  f51  CM)  - f52  CM)  f53  Ch)  2(Hi7)  TT“  91 

- [f54  CM)  + fss  Ch)  T56  CM)]  5is  >. 

Ka  : Contribution  to  pitching  moment  from  rate  of  change:  of  angle  .... 

. of  -attack,  >•  7-^:,/":- ^rv'"*-  -r:.r  •;  "• 

Kq  : Contribution  of  thrust  to  pitching  moment*  —.-••• 


(from.  Wilke r son,  Norman,  Matheny,  Demaree  and  Lowes , 1965) 
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TABLE  24.  PERCENTAGE  CHANGES  IN  LONGITUDINAL  PARAMETERS 
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TABLE  25.  PERCENTAGE  CHANGES  IN  LATERAL  PARAMETERS 
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instrument  readings,  platform  motion  and  visual  displays).  In  this  para- 
graph, controls  are  discussed  in  more  detail  by  reviewing  several  classes 
of  controls,  the  mathematical  relations  that  describe  their  action  and  some 
of  the  facts  known  about  the  sensitivity  of  man  in  operating  the  more  im- 
portant types  of  controls.  Kelley  (19  68)  subdivides  controls  into  "force 
operated"  controls  and  Mpos itioningM  controls.  Some  of  the  important 
differences  between  these  two  types  of  controls  are  described  by  Kelley: 


Force  Operated 


Position  Operated 


1.  Control  forces  correspond 

to  forces  applied  by  operator; 
"natural"  control. 


1.  Control  forces  do  not  correspond 
to  forces  applied  by  operator; 
interpretive  step  required  for 
control . 


2.  Control  lever  is  self-centering;  2.  Control  lever  remains  at  position 
forces  diminish  to  zero  unless  on  which  placed;  forces  remain 

manual  force  is  maintained  on  applied  without  maintaining 

the  control.  manual  force  on  the  control. 


3.  A large  range  of  forces  may  be 
accurately  controlled  in  a small 
range  of  control  lever 
displacement. 

4.  To  control  a large  range  of 
forces  accurately,  large 
amounts  of  manual  force  are 
required. 


3.  To  control  a large  range  of 
forces  accurately,  a large  range 
of  control  lever  movement  is 
needed. 

4.  A large  range  of  forces  can  be 
controlled  accurately  with  very 
small  manual  forces  . 


5.  Because  of  (4)  , to  control  a 5 . Because  of  (4)  , a large  range  of 

large  range  of  forces  accu—  forces  can  be  controlled,  accu- 
rately, a control  must  be  rately  by  many  types  of  controls, 

built  and  located  so  the  involving  very  slight  forces , and 

operator  may  exert  large  placed  in  a large  range  of 

manual  forces  on  it . locations. 


Free  positioning  controls  usually  maintain  a set  position  by 
virtue  of  sliding  friction,  and  the  operator  senses  the  position  of  his  body 
members.  The  forces  required  to  move  the  control  are  too  small  to  result 
in  significant  feedback.  Force  operated  controls  requiid  greater  force  and 
the  feeling  of  the  applied  pressure  provides  useful  feedback.  The  force 
operated  c ontrols  may  be  sub divi.de d into  se ve r al  groups  . For  a rigid  type 
of  control  with  strain  gage  pickup,  the  feedback  to  the  operator  is  simply 
the  pressure  resulting  from  the  amount  of  force  he  applies.  ■ For  a simple 
spring  loaded  type  of  control,  motion  is  proportional  to  the  amount  of  force, 
and  the  operator  senses  both  pres  sure  and  position.  Viscous  controls  move 
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at  a velocity  proportional  to  the  applied  force,  and  they  remain  displaced 
when  the  applied  force  is  zero.  Finally,  inertial  controls  accelerate  at 
a level  proportional  to  the  applied  force,  and  when  the  force  is  removed 
they  continue  to  move  at  a constant  rate.  Each  of  the  four  categories  of 
force  operated  controls  may  be  considered  as  idealized  categories.  Any 
actual  aircraft  or  simulator  force  control  may  be  considered  as  a com- 
bination of  four  types-  and  the  operator  senses  force  level,  position, 
velocity  and  acceleration.  If  we  take  X as  the  displacement  of  force  con- 


trol, resulting  from  an  applied  force,  f,  then  the  following  relationships 
apply: 


X = k-^f,  for  the  spring  loaded  case 


X = k2f., 


for  the  viscous  case 


X ='  k^f,  for  the  inertial  case 


where  k^V'k^*  and  k^  are  constants  of  proportionality 


The  control,  whose  response  is  not  purely  inertial 
or  spring  loaded,  may  be  described  by  the  relation 


, viscous 


f = a + bX  + cX  + dX 


where: 


f is  the  force  exerted  by  the  control 


X is  the  displacement  for  some  arbitrary  reference  position 


X is  the  velocity  of  the  control 


X is  the  acceleration  of  the  control 


a is  the  force  exerted  at  the  reference  position 


values  of  a , b , c , and  d are  t 
c o nditio n s of  s imulate  d f li ght 
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Up  to  this  point,  we  have  been  discussing  the  simple  case 
where  a control  is  fres  toJmove  in  one  direction.  The  above  relationship 
may  be  extended  in  a straightforward  way  to  describe  more  complex  con- 
trols, e . g . , a yoke  . Thus  , . 

f — a -f~  bX  + c]t  -1  d3i  +e  o + f ct  + g 
where  most  of  the  terms  are  as  previously  defined,  but 

a is  the  rotational  displacement  of  the  control  from 
an  arbitrary  reference  position 

a is  a rotational  velocity  of  the  control 

tf  is  the  rotational  acceleration  of  the  control 

a is  the  force  exerted  at  the  reference  position  for 

displacement  and  rotation.  • , . 

Although  the  two  yokes  may  feel  the  same  if  their  values  of  a,  b . . . g are 
the  same  under  different  flight  conditions,  the  designer  is  faced  with  the 
inevitable  question  of  how  much  deviation  can  be  tolerated  without  being 
noticed  by  the  trainee.  The  designer  is  interested  in  the  question  of 
psychophysical  discrimination  for  two  reasons.  First,  he  would  like  to 
know  "what  he  can  get  away  with11  when  trying  to  duplicate  an  aircraft’s 
controls.  Second,  he  would  like  to  know,  how  much  control  force  enhance- 
ment is  required  in  developing  the  necessary  training  cues. 

3.  3. 2.4.4  Force  Levels.  Pilot  control  forces  in  realistic  situations  are 

difficult  to  duplicate  in  the  laboratory.  However  , some  guidance  concerning 
perceptually  significant  differences  in  control  force  levels  is  provided  from 
the  work  of  Jenkins  (1 947)  . O';4  . 

Difference  Lime  ns'  for  20  Pilots  Reproducing  Aircraft  Control  Forces 


Standard 


Pressure  Stick 

(lbs)  Control 


1 .20 

5 .10 

10  .08 

20  Jv  H . 08 

30  .07 

40  .06 


Wheel  Rudder 

Control  Pedals  ■.», 


2 3 

- 

09 

. . 10 

0 7 

.07 

06 

05 

06 

, .05 

0 5 

•'  '.-V  ; 04 

- 287  - 


NAVTRADEVCEN  69-C-0298-1 


He  found  that  the  difference  limens  or  Weber  fraction  (standard  deviation 
divided  by  reference  force)  was  about  .20  at  the  one  pound  force  level  and 
decreased  to  .07  at  the  10  lbs.  force  level.  About  10  lbs.  of  force,  the 
Weber  fraction  remained  at  about  .06.  Jenkins  also  found  that  just  resting 
a hand  on  a control  was  equivalent  to  about  1 lb.  of  force,  and  resting  a 
foot  on  a simulated  rudder  yielded,  a force  of  5 lbs.  which  implies  that  dis- 
crimination of  forces  below  those  levels  should  be  avoided  in  design  con- 
siderations . 


3. 3.2.4. 5 Effective  Time  Constant.  The  effective  time  constant,  te , 

is  a useful  construct  to  compare  simulator  and  aircraft  controls.  It  is  also 
a useful  measure  on  which  to  base  simulator  control  modifications  during 
the  training  period,  since  te  is  related  to  task  difficulty.  This  construct 
was  used  by  Matheny  and  Norman  (1968)  to  describe  the  delay  in  speed  of 
feedback  to  an  operator  who  has  made  a control  input.  The  effective  time 
constant  describes  the  entire  man-machine  system  but  is  strongly  influ- 
enced by  such  control  factors  as  gain  and  damping  parameters.  Matheny 
and  Norman  (1968)  define  te  as  ".  . .the  time  interval  from  the  initiation  of 
a one-inch  step  displacement  of  the  control  stick,  measured  at  the  top  of 
the  stick,  to  the  time  at  which  the  display  member  has  been  displaced 
through  .4  degrees  of  arc  from  its  initial  position,  measured  at  the  position 
of  the  operator's  eye."  (p.  2.) 

Regardless  of  the  system  order,  the  value  of  te  is  affected 
by  a number  of  different  factors.  A first  order  system  may  be  described 
by  the  relationship: 

yQ(t)  = * (1  - e -t/  T) 

where:  y0(t)  is  system  output  as  a function  of  time  > 

k is  system  steady  state  gain 

t is  time-  A. .#  : -1.?: ' _ : r ■ . ■■  ■■ 

T is  the  time  constant  : 

_t  ' is  affected  by  (or  may  he  controlled  by)  tiie  valixes  of  t 9 and  the 

threshold  value  of  yQ(t). 

A second  order  system  may  be  described  by  the  equations: 

>Q(t)  = \K.  [* 

d =Vi 


i 

d 


sm  ( wdt  - arc  tan 


-] 


o 
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where:  yQ(t)  is  system  output  as  a function  of  time 

tc  is  system  steady  state  gain 
co  is  the  undamped  natural  frequency 
£ is  the  damping  ratio 
t is  time 

d is  the  damping  factor,  defined  above  as  a function  of  £ . 

From  the  second  order  system  equations,  it  is  apparent  that  te  is  controlled 
by  k , o»  and  5,  in  addition  to  the  threshold  value  of  yQ(t)  . 


Matheny  and  Norman  (1968)  observed  that  lower  values  of 
te  resulted  in  an  easier  closed-loop  task.  The  relationship  seemed  linear 
between  te  and  performance  when  a logarithmic  transformation  was  used 
for  te  . They  also  observed  that  for  untrained  people  the  prcduct  <te  was 
directly  related  to  task  difficulty.  That  is,  for  a given  level  of  te  , lower- 
ing the  value  of  Krte  simplified  the  tracking  task  for  unskilled  per  sonnel. 


The  . designer  may  therefore  wish  to  have  the  system  gain, 
natural  frequency  and  damping  ratio  (and  hence  te)  controllable  , and  often 
different  from  that  of  the  aircraft  simulated.  To  assist  the  transfer  of 
training,  the  values  of  < and  te  might  be  set  lower  than  the  "true"  levels  . 
Later,  as  skill  improved,  the  values  could  be  raised  to  normal  levels  or 
possibly  to  above -normal  levels. 

3.3.3  Evaluation  Criteria.  This  paragraph  summarizes  points  already 
discussed  and  may  serve  as  a checklist  for  the  designer  in  evaluating  a 
set  of  simulator  controls  (i.  e . , when  the  more  fundamental  bounds  of  the 
design  effort  have  been  established).  In  evaluating  the  simulator  controls, 
the  following  points  should  be  considered: 


Are  the  controls  designed  with  the  intent  of  optimizing 
training,  rather  than  merely  duplicating  the  air cxaft1  s 
controls  as  closely  as  possible? 


Gan  the  controls  be  adjusted  to  make  flying  the  simulator 
easier  than  flying  the  aircraft  by  adjusting  such  things  as 
gain,  effective  time  constant,  enhancement  of  natural 
cues  , etc  . ? 


Are  the  controls  capable  of  supplying  "artificial"  cues 
to  the  trainee? 
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• Within  their  performance  envelope,  are  the  controls 
capable  of  duplicating  those  of  the  aircraft  with  a 
high  degree  of  perceptual  fidelity  to  help  motivate 
the  advanced  student?  Some  of  the  control  param- 
eters to  be  considered  are  control  force,  position, 
viscous  and  inertial  response,  gain,  and  effective 
time  constant.  (For  most  of  these  parameters  the 
designer  will  have  to  use  judgment  as  to  what  is 
"close  enough"  to  reality;  Tables  24  and  25  indicate 
the  kinds  of  errors  in  the  aerodynamic  equation 
parameters  which  may  be  tolerated). 

• Are  the  controls  amenable  to  adaptive  training  techniques? 

• Are  the  controls  configured  like  those  of  the  simulated 
aircraft;  can  the  controls  be  readily  reconfigured  to 
duplicate  different  versions  in  the  aircraft  design? 

• Are  the  controls  designed  to  minimize  total  training 
costs,  i.  e.  , simulator  design  and  fabrication  costs  as 
well  as  training  time  costs  ? 

3.3.4  Research  Issues.  Several  research  issues  of  priority  for  human 
factors  design  are  indicated  next.  These  concern: 

improving  responses  to  natural  cues 

application  of  artificial  cues 

adaptive  training 

cost  modelling  development. 

a)  Some  effort  is  desirable  on  understanding  the  enhancing  effects 
of  natural  cues.  Skilled  pilots  often  make  use  of  cues  which  are  not  re- 
sponded to  by  unskilled  trainees  . Thus  , pertinent  questions  for  design  are, 
which  control  characteristics  should  be  heightened,  when  and  to  what  extent, 
and  when  in  the  training  cycle  should  the  augmentation  be  discontinued.  The 
proprioceptive  senses  are  especially  sensitive  to  changes  in  level  rather 
than  to  the  absolute  magnitude  of  level,  hence,  study  is  suggested  to  deter- 
mine training  benefits  from  transformation  of  stick  forces  (or  displacements) 
which  would  heighten  naturally  occurring  changes  . One  possible,  trans  - 
formation  suggested  is  illustrated  by  the  equation: 

V,;  .;  : V ; •• 
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where:  ds  = is  the  displacement  (or  force  level)  of  the  simulator  control. 


d|.  = is  the  “true"  displacement,  if  augmentation  were  not 
incorporated, 

A = the  cue  augmentation  factor, 

B,C  = factors  chosen  to  Mwash  out11  the  effect  of  A on  dg  after  a 
short  period  of  time . 

b)  Another  research  issue  of  concern  is  the  relationship  of  artifi- 
cial control  cues  to  the  feedback  of  knowledge  of  performance  to  the  trainee. 
Reference  is  made  here  to  the  use  of  controls  which  provide  signals  , e . g.: 
vibrate,  as  a trainee’s  maneuver  deviates  from  an  acceptable  performance 
envelope;  or  which  are  difficult  to  operate  in  marked  difference  to  the 
programed  optimum. 

c)  The  issue  of  adaptive  training  is  also  of  some  concern  to  vehicle 
control  (see  Chapter  3.6).  The  recommendation  is  to  explore  systematically 
the  adjustment  of  task  difficulty  by  modifying  simulator  control  response 
(rather  than  task  modification  via  adjustment  of  a forcing  function,  such  as 
turbulence).  This  research  would  examine  the  adjustment  of  level  of  arti- 
ficial cues  or  level  of  natural  cue  enhancement  (e.g.  , lag  or  sensitivity)  in 
accordance  with  trainee  performance. 

d)  Although  not  directly  related  to  this  chapter , much  research  is 
needed  towards  establishing  basic  limits  on  training  device  design.  There 
is  a mounting  pressure  to  examine  the  tradeoffs  that  may  bemade  concern- 
ing training  time  in  the  aircraft  vs.  training  time  in  the  simulator.  One 
avenue  suggested  is  cost  modeling  studies  to  investigate  the  functional  . 
relationships  between  training  value  in  the  aircraft  and  in  the  simulator  . 

One  concern  is  the  question,  can  training  time  in  a simulator  and  in  an 
aircraft  be  traded  off  linearly  , i.  e . , k^  hours  on  a simulator  for  one  hour 
in  an  aircraft?  If  so,  what  are  the  limits  under  which  this  tradeoff  may 

be  made  , and  what  is  the  value  of  k£?  Similarly  , can  the  same  kind  of 
tradeoff  be  made  for  training  time  in  a sophisticated  simulator  and  in  a 
reduced  fidelity  simulator  , or  in  a simulator  and  in  a relatively  inexpensive 
trainer?  In  order  to  make  decisions  on  where  design  and  fabrication  costs 
can  best  be  reduced,  some  idea  is  needed  on  the  nature  of  the  function  ITas 
and  how  it  changes  with  different  types  of  simulators  and  aircraft. 

Illustrative  of  this  concern  for  setting  basic  limits  on  simulator 
design,  a simple  cost  model  of  the  training  situation  is  outlined  below: 

G = As  i OsTs  + Aa  + OaTa,  ; t ~~  . 
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where  C is  the  total  cost  of  achieving  an  objective  training  goal 


As  is  the  acquisition  costs  of  the  simulator  (including  design 
and  fabrication  costs) 

Os  is  the  operating  costs  of  the  simulator  (including  maintenance 
and  the  costs  of  running  the  simulator  during  training  sessions) 

Ts  is  the  time  spent  on  operating  the  simulator 

Aa  is  the  acquisition  costs  of  the  aircraft  for  training  purposes 
Oa  is  the  operating  costs  of  the  aircraft 


Ta  is  the  time  spent  on  operating  the  aircraft 


It  should  be  noted  that  this  cost  model  deals  with  objective  costs. 
For  example  , it  doesn't  include  the  safety  value  of  initial  training  on  a 
simulator.  The  model  also  presupposes  that  the  attainment  of  the  training 
goals  can  be  determined  in  some  objective  way,  i.e.,  successful  accom- 
plishment of  training  objectives.  Thus,  for  a given  simulator  and  aircraft, 
Ag,  Os  , Aa  and  Oa  are  fixed.  Ts  may  be  considered  an  independent  vari- 
able while  Ta  is  a dependent  variable  , whose  length  is  sufficient  to  achieve 
the  desired  training  goal  with  a given  simulator  and  aircraft.  The  precise 
form  of  Fas  would  have  to  be  determined  empirically.  However,  to  graphi- 
cally illustrate  the  effect  of  Fas  , Figure  36  shows  the  effect  of  using  differ- 
ent values  of  Ts.  Time  is  shown  on  the  horizontal  axis  and  cost  on  the 
ordinate.  For  minimum  cost  the  distance  between  the  origin  and  repre- 

sents Ts,  while  Ta  is  represented  by  the  distance  t2  - tp  Os  is  Cp  Oa  is 
C2  - Cl.  The  minimum  total  cost  is  C4.  For  the  minimum  cost  case, 

C3  - C2  is  the  variable  simulator  cost,  and  C4  - C3  is  the  variable  aircraft 
cost.  A total  cost  which  is  non-minimum  (because  of  insufficient  simulator 
time)  is  shown  at  C5.  The  slope  of  the  line  through  C2  is  Os ; while  the 
slope  of  the  dashed  line  s is  Oa . 

For  the  purpose  of  discussion,  we  may  assume  a simple  analytic 
representation  of  Fas  . The  form  selected  is  linear  and  thus  even  more 
simple  than  the  graphic  representation  in  Figure  36,  viz: 

Ta  .=  ki  - Ts/k2  , when  Ts  is  below  (-kj;  - k3>k2 

Ta  - k3 , otherwise 


This  is  equivalent  to  saying  that  the  training  task  would  take  kj  hours  if  the 
simulator  were  not  used  at  all;  at  least  k3  hours  must  be  spent  in  the  air- 
craft, regardless  of  the  time  spent  in  the  simulator;  and  k'2  hours  in  the 
simulator  are  equivalent  to  one  hour  in  the  aircraft.  For  this  simple  form 
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Time  ^ 

C1-0  Edjxed  simulator  operating  cost 

Fixed  aircraft  operating  cost 

C3-C2  Variable  simulator  cost” for  minimum  total  cost 

C^"C3  Variable  aircraft  cost  for  minimum  total  cost 

C4-0  Minimum  total  cost 

C -0  A non^minimum  total  cost 

t^-0  Time  in  simulator  for  minimum  total  cost 

t^-0  Time  in  aircraft  for  minimum  total  cost 


Figure  3 6.  Illustrative  Total  Cost  Curve 
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°f  Tas  , if  it  pays  to  use  a simulator  at  all  (essentially  if  <Oa)  , it  would 

pay  to  have  the  trainee  spend  (kl  - k3)k2  hours  in  the  simulator.  The  mini- 
mum cost  of  reaching  the  training  objective  would  be  as  follows: 

cmin  = As  + Aa  = k3°a  + (kl  - k3)k20s 

From  the  point  of  view  of  the  simulator  designer  the  question  of 
whether  it  pays  to  add  more  realism  can  be  considered  analytically  in  terms 
of  the  effects  on  the  C rnin  equation.  More  realism  would  probably  tend  to 
increase  Cmin  due  to  the  changes  in  As  and  Os  but  tend  to  decrease  Cmin 
as  a result  of  the  changes  in  k2  and  k3  . Depending  on  the  magnitude  of  the 
alternative  values  of  As,  Os  , k2  and  k3  , Cm^n  can  be  calculated  as  raised 
or  lowered  by  the  increased  realism.  Quite  obviously,  if  the  contemplated 
change  improved  physical  fidelity  but  not  perceptual  fidelity,  only  As  and 
Os  would  be  affected  and  would  increase. 


Work  is  required  to  develop  this  cost  modeling  approach,  to  ascer- 
tain functional  forms,  parameter  magnitudes  and  the  like. 
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3.4  INFORMATION  PROCESSING 


Man./ machine  system  performance  is  importantly  related  to  the  human 


ability  to  detect  and  to  process  relevant  input  information.  Man's  capacities 
are  not  limitless  since  he  is  subject  to  the  constraints  imposed  by  the  nervous 
system  and  musculature  as  a function  of  the  job  requirements.  Thus,  knowl- 
edge of  human  capabilities  in  processing  information  is  of  importance  to  the 
design  for  training,  particularly  for  optimizing  design  in  terms  of:  what  man 

can  effectively  do  or  handle  in  the  time  domain;  how  he  processes  information; 
and  the  display  characteristics  most  suited  to  his  capabilities. 

Viewing  man  as  a single-channel  limited  transmission  capacity  infor- 
mation processing  system  provides  the  potential  for  specifying  system  com- 
ponents most  compatible  with  his  capabilities  . In  sinipls  terms,  environ- 
mental stimuli  impinge  on  man's  sensory  systems,  a complex  set  of 
transformations  are  applied  and  a motor  output  is  transmitted  via.,  thp 
response  system.  The  limited  capacity  of  the  single  channel,  implies  that, 
when  simultaneous  activities  each  make  demands  on  the  single  channel  the 
performance  of  one  will  be  at  the  expense  of  others,  i.  e.  , \tjisko 

must  time  share  the  single  serial  channel . ■*-  This  view  of  huiriafi  perfor- 
mance attempts  to  describe  in  quantitative  terms  the  nature  o;  the  transfor- 
mations of  input  signals  into  outputs  as  a function  of  the  description  of  the 
environment,  the  nature  of  the  motor  response  and  the  transformations  that 


are  required  (Few,  1965)  . 3 

matized  as  follows: 
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Block  1 embraces  the  processes  of  sensation  and.  perception.  The  sensory 
system  of  man  transduces  the  inputs  and  codes  them  into  patterns  of  neural 
excitation  from  which  decisions  can  be  made  or  stored  in  memory.  Man’s 
perceptual  system  is  of  finite  capacity  and  incapable  of  analyzing  simultane- 
ously all  information  received  by  the  sense  organs.  Only  part  of  the  total 
stimulation  impinging  on  the  receptors  is  capable  of  initiating  a response. 
Thus,  the  issues  of  what  man  attends  to  in  displays  particularly  in  high 
information,  complex  formats,  and  how  much  information  can  he  assimilate, 
organize  and  respond  to  are  of  considerable  concern  to  the  human  factors 
design  of  training  devices.  Block  2 concerns  the  memory  component  which 
subsumes  relatively  permanent  long-term  storage  and  impermanent  short- 
term storage.  The  digital  computer  analogy  is  to  the  random  access  core 
memory  and  the  buffer  storage,  respectively.  The  creation  of  information 
occurs  when  memory  is  used  to  elaborate  upon  a stimulus  . Broadbent 
(1958)  has  hypothesized  that  incoming  signals  are  directed  to  the  buffer 
storage  from  whence  they  can  be  further  processed  or  stored  in  memory  . 
Block  3 is  concerned  with  man’s  abilities  in  the  manipulation  and  combina- 
tion of  input  and  stored  data.  In  this  decision-making  subsystem,  the 
fundamental  limitations  on  the  rate  of  information  processing  occur  , For 
the  purposes  of  this  report,  we  are  most  interested  in  the  operations  of  this 
subsystem,  for  knowledge  of  how  a man  performs  as  a function  of  task 
makeup  and  how  much  of,  and  how  fast  he  can  accomplish  job  requirements 
depends  on  information  handling  rates.  Block  4 is  concerned  with  the  trans- 
lation of  the  outputs  of  the  information  processing  subsystem  into  observable 
actions  which  effect  adjustments  to  the  environment.  The  limitations  in 
man’s  information  outx^ut  rates  are  much  less  severe  than  the  limitations 
imposed  in  Block  3. 

a.  Types  of  Human  Information  Processing 

Quantitative  prediction  of  information  processing  rate  depends 
on  task  characteristics.  Three  forms  of  information  processing  tasks  have 
been  identified.  These  are  tasks  involving  the  conservation  of  information, 
tasks  involving  information  filtering  and  tasks  involving  the  condensing  or 
reducing  of  information  (Pew,  1965;  Fitts  and  Posner,  1967). 


Conservation  of  information  tasks  are  defined  by  the  feature  that 
all  the  information  in  the  input  signal  is  retained  in  the  output,  thus  the 
stimulus  can  be  inferred  precisely  from  the  response  . In  these  tasks  where 
the  human  is  a relay  (e.  g.  , typewriting  is  typical)  the  consistently  relevant 
variable  influencing  performance  rate  is  the  amount  of  information  contained 


in  the  input.  Other  variables  influencing  the  rate  of  processing  include: 
stimulus  - response  compatibility ; the  disc  riminability  of  input  codes;  and 
uncertainty  for  when  the  next  signal  will  occur  . , feV < : \ 


Information  filte  ring 


tor ing/ vigilance  performance 


tasks  characteristically  concern  the  moni- 


of  man  wherein  he  is  required  to  examine  the 
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status  of  identified  variables  and  respond  only  when  the  value  of  a variable(s) 
is  ixi  some  defined  critical  state.  In  this  type  of  task  the  human  is  utilized 
as  a monitor. 


Information  condensing  tasks  impose  the  requirement  for  the 
collapsing  or  reduction  of  input  information  into  a smaller  set  of  output 
classes,  i.e.  , the  amount  of  information  in  the  output  is  considerably  less 
than  contained  in  uhe  input  signals  . This  type  of  task,  situation,  where  the 
human  functions  as  an  evaluator  is  of  particular  importance  to  design  in 
that  many  of  the  activities  performed  in  weapons  systems  involve  informa- 
tion reduction  in  o7*.ie  way  or  another.  Information  is  combined  from  vari- 
ous sources  or  from  a single  source  with  the  result  being  a single  output 
operation  or  decision.  In  fact,  filtering  and  condensing  requirements  are 
prominent  in  visual  display  interpretations,  such  as  the  sonar  classification 
task,  and  Radar  and  EW  scope  analyses  having  high  density  signal  and  noise 
relationships.  Information  reduction  is  also  the  hallmark  of  tactical  de- 
cision-making activities. 

Our  concern  in  this  section  centers  on  human  information  pro- 
cessing capacity  as  this  relates  to  the  design  of  training  hardware  . We 
will  attempt  to  select  from  the  sizeable  body  of  research  on  information 
processing  and  handling  those  concepts  and  data  pertinent  to  the  selection 
of  design  alternatives.  Owing  to  practical  limitations  , no  attempt  is  made 
to  organize  systematically,  data  on  sensory  input  variables,  on  the  mem- 
ory store  subsystem  or  on  the  response  subsystem  (translation  of  processed 
information  into  observable  actions).  Each  is  a complex  story  in  itself  and 
each  is  well  represented  in  the  published  literature. 

3.4.1  Human  Information  Handling  Capacity.  Information  handling  capa- 
cities of  the  operator  are  definable  in  terms  of  speed  and  accuracy  of  task 
performance.  In  the  framework  which  regards  man  as  an  information 
channel  this  is  defined  in  terms  of  the  amount  of  information  transmitted 
per  unit  of  time,  i.  e.  , the  number  of  binary  decisions  (bits)  involved  in 
information  processing.  This  suggests  that  the  operator  transmits  infor- 
mation at  a rate  that  is  dependent  on  the  particular  task.  When  more  infor- 
mation is  transmitted  in  a given  task,  a longer  time  is  required. 

As  an  information  link , man  is  capable  of  processing  about  3 bits 
of  information  per  stimulus  event  for  any  stimulus  dimension.  Thus  , in 
perceptual  terms  , man  is  able  to  identify  absolutely  from  six  to  ten  steps 
along  any  physical  dimension.  Table  26  indicates  the  capacity  for  absolute 
judgment  along  a single  dimension.  Hick  ( 1952)  , based  on  his  studies  , 
advanced  the  finding  that  response  time  varies  as  a logarithmic  function  of 
the  number  of  alternative  stimulus  events  . The  rate  of  gain  of  information 
is  constant.  Transmission  rate  tends  toward  constancy  regardless  of  .. 

changes  in  the  presentation  rate  or  information  content  of  the  stirnuli. 
Figure  37  depicts  the  relationship  between  information  processing  time 
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table 

26.  CAPACITY 
A SINGLE 

FOR  ABSOLUTE 
DIMENSION 

JUDGMENT  ALONG 

Modality 

Dimension 

Maximum 

Information 

Transmitted 

(Bits) 

Approximate 
Number  of 
Stimuli  Correctly 
Identified 

Vision 

Position  on 
a line 

3. 3 

10 

Vision 

Hue 

3.1 

9 , 

Audition 

Pitch 

2 . 5 

6 

i ■ ■: 

■ '-V 

Audition 

Loudness 

2.3 

5 

Taste 

Saltiness 

1.9 

4 

Data  taken  from  Attneave  ( 1 c,  5 9 ) . 


and  amount  of  information  transmitted,.  It  appear?  that  man  can  trade  off 
speed  for  accuracy  and  vice  versa,  so  that  if  he  is  required  to  respond 
more  rapidly  or  must  attend  to  a greater  number  of  stimuli  at  the  same 
rate,  accuracy  will  decrease  proportionately  so  that  the  same  amount  of 
information  will  be  transmitted. 

Although  the  relationship  of  speed  and  accuracy  tradeoffs  has  been 
demonstrated  in  maintaining  the  'constancy  in  rate  of  gain  of  inforrhation, 
other  variables  will  alter  the  relationship  somewhat  . Pew  (1965)  has  sum- 
marized a number  of  variables  that  have  been  shown  to  affect  the  rate  of 
information  transmission  especially  in  information-conservation  tasks  . 

The  variables  which  are  important  to  design  include  the  following: 

. The  compatibility  of  the  relationship  between  the  input 
code  and  the  output  code  is  an  ^important  determinant 
of  processing  rate  (the  issue  of  stimulus — response 
corhpatibility  is  discus  sed  later  in  this  chapter)  . 

m The  rate  of  processing  will  be  decreased  if  the  input 

signal— code  symbols  are  less  than  perfectly  disc 'riminable. 
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Figure  37  . Data  illustrating  the  linear  relation  between. 

information-processing  time  and  amount  of  v 
information  transmitted  in  a key-pressing 
task.  The  two  curves  represent  two  sets  of 
data,  one  obtained  by  Merkel  in  1885  and  the 
other  by  Hick  in  1952  . The  slightly  different 
slopes  and  intercepts  of  the  two  curves  reflect 
the  effects  of  different  stimulus  and  response 
• codes  used  in  the  two  experiments  :r  - • 


(from  Few, 


1965) 
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• The  operator's  task  set  concerning  which,  input  signals 
will  occur  and  when  they  will  occur,  and  the  relative 
importance  of  speed  and  accuracy  on  performance, 
affect  information  processing  rate. 

• Increasing  the  uncertainty  as  to  when  the  next  signal 
will  occur  has  the  effect  of  delaying  the  processing 
time  . 


For  information  filtering  tasks  (i.e.  , where  man  functions  as  a 
monitor  of  system  events)  the  processing  time  is  affected  when  the  rate 
of  information  presentation  is  slow  and  the  critical  vs.  no  response  signals 
are  significantly  different.  However,  when  the  critical  and  the  non-critical 
signals  are  similar  (e.g.  , needle  positions  on  a graduated  gauge)  , then 
processing  time  per  signal  increases.  It  has  been  demonstrated  that  the 
number  of  critical  states  searched  for  in  a given  signal  can  be  as  many  as 
ten  without  a substantial  increase  in  processing  time  per  signal  so  long  as 
a single  response  is  common  to  all  the  critical  aspects  of  a signal  (Neisser, 
1963) . I • 

The  most  important  variable  in  information  reduction  tasks  is  the 
amount  of  information  reduced  in  translating  the  input  into  the  output.  The 
amount  of  information  reduction  required,  that  is,  the  difference  (in  bits) 
between  the  input  information  and  the  appropriate  response  information 
shows  a linear  relation  to  the  time  required  to  accomplish  the  reduction 
task.  The  greater  the  reduction  the  slower  is  the  processing  (Posner, 

1965). 


3.4.2  Variables  Affecting  the  Perceiving  and  Processing  of  Information. 

A series  of  questions  important  to  trainee  station  design,  particularly  to  the 
displays  for  training,  concerns  such  issues  as,  what  does  the  man  attend  to 
in  the  multi -signal  and  noise  environment,  and  how  does  he  attend  to  input 
stimuli  since  not  all  stimuli  are  equivalent?  Allied  to  these  is  the  consi- 
deration of,  under  what  circumstances  does  man  process  information  in 
certain  ways?  The  data  available  on  how  man  attends  to  and  processes  in- 
formation has  utility  for  design,  particularly  as  an  assist  to  such  deter- 
minations as:  how  much  can  a man  accomplish  in  various  tasks  and  task 

conditions  per  unit  of  time;  how  much  time  does  he  require  to  complete 
certain  activities ; how  does  he  search  visual  displays;  what  is  his  attention 
span;  how  does  he  organize  and  utilize  high  density  displayed  information 
including  mor  e than  one  display  with  or  without  multi -formats  ? Knowledge 
about  such  operations  will  assist  in  the  design  of  displays  and  in  the  pre- 
sentation of  own- vehicle  / target  and  media  information. 

The  view  of  man  as  a single  channel  information  processor  assumes 
that  he  must  time  - share  among  competing  stimuli . Thus  , the  variables  in- 
volved in  attention  are  of  considerable  interest  for  human  factors  design. 


ri 
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Two  aspects  of  attention  are  involved:  selective  attention  (how  man  attends 

to  or  notices  particular  parts  of  the  displayed  environment);  and  sustained 
attention  (vigilance  behavior  — monitoring  events  of  infrequent  occurrence). 
For  selective  attention,  the  nature  of  the  stimulus  input  is  an  important 
variable  in  determining  how  man  processes  information.  Since  all  stimuli 
are  not  equivalent,  selective  attention  is  a function  of:  1)  the  sense  modality, 

2)  the  signal  characteristics  (frequency,  intensity,  spatial  location,  tem- 
poral characteristic  s , etc.),  and  3)  the  display  characteristics  (stimulus 
enhancement,  coding,  organization  of  data,  etc.).  Sustained  attention  is  a 
function  of  the  minimum  input  characteristics  and  temporal  relations. 

In  the  remainder  of  this  subsection,  design  variables  which  affect 
the  detection  and  the  processing  of  relevant  input  information  are  discussed. 


3.4.2.  1 Detecting  and  Discriminating  Visual  Signals.  An  aspect  of 
skilled  performance  is  the  ability  to  detect  and  identify  a target  from  an 
array  of  stimuli  or  events  . Questions  concerning  the  manner  in  which  man 
performs  search  and  detection  tasks  or  what  man  attends  to  in  a multi - 
signal  environment  are  of  significance  for  design,  for  example,  specifying 
the  number  of  targets  to  display  simultaneously  or  specifying  the  display 
formats  in  order  to  manage  the  difficulty  levels  of  a task  for  training. 


In  visual  displays  , the  operator  must  rapidly  and  accurately 
locate  and  identify  relevant  signals  or  events.  In  the  visual  search  task 
he  must  simply  scan  to  detect  the  presence/ location  of  specific  signals  . 

In  the  identification  task  he  must  decode  the  presented  information.  In 
each  case  the  structural  characteristics  of  the  display  significantly  effects 
performance,  and  dimensions  such  as  shape,  size,  brightness , and  hue , 
affect  the  search  and  decoding  time . Generally,  search  time  is  increased 
as  the  number  of  irrelevant  signals  is  increased.  For  example,  Nickerson 
and  Duva  ( I960)  , examined  the  discrimination  of  discrete  dot  positions  in 
a display  as  a function  of  exposure  time,  display  size  and  dot  size.  They 
found  that  the  time  available  for  viewing  the  display  to  be  most  critical. 

For  an  exposure  time  of  10  seconds,  dot  position  in  an  imaginary  8X8 
matrix  was  judged  with  almost  perfect  accuracy . Sixty-four  discrete  posi- 
tions of  a dot  within  a square  could  be  satisfactorily  discriminated  for  10  : 

second  exposure  times. 


Concerning  statistical  aspects  of  signal  identification,  Howell 
and  Briggs  ( 1959)  report  on  laboratory  studies  in  which  nonsense  patterns 
(constructed  according  to  statistical  specifications)  were  used  as  stimuli. 
Subjects  were  required  to  identify  figures  which  were  altered  in  certairl 
parameters  . The  finding s suggest  the  following  when  designing  visual? 
stimulus  figures  to  enhance  recognition: 

r;  Recognition  is  superior  for  figures  constructed 


O 
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via  a random  process  vs.  biased  procedures  for 
obtaining  nonsense  forms  . 
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• Symmetrical  figures  are  more  easily  recognized  than 
a symmetrical  figures. 

• Redundancy  in  figure  construction  improves  perfor- 
mance when  figures  are  degraded.  (This,  however, 
may  be  detrimental  under  ideal  conditions.)  Re- 
dundant information  is  helpful  in  stimulus  recognition 
when  noise  or  other  distortions  will  occur  in  the 
situation.  The  use  of  redundant  dimensions  is  ad- 
visable when  accuracy  is  more  important  than  speed 
of  when  the  frequency  of  signal  happenings  is  low. 

a Hue  is  a more  easily  identified  dimension  than  size 
or  brightness  in  signal  recognition. 

• The  operator  utilizes  only  the  amount  of  information 
required  by  the  recognition  task:;  superfluous  infor- 
mation is  detrimental. 

» In  tasks  containing  noise,  a greater  amount  of  infor- 
mation is  necessary  for  successful  stimulus  recogni- 
tion than  in  noise-free  tasks. 

An  aspect  of  selective  attention  is  the  ability  to  search  for  and 
identify  a target  from  a stimulus  array.  In  simple  laboratory  task  situa- 
tions , the  relationship  between  the  total  number  of  items  to  be  searched 
and  the  time  required  to  find  the  target  item,  is  linear  (Fitts  and  Posner, 
1967).  When  items  to  be  searched  are  letters  of  the  alphabet,  the  slope 
of  the  function  is  about  100  milliseconds  per  item  (Neisser,  et  al,  1963). 
The  rate  at  which  items  are  searched,  however,  depends  upon  the  nature 
of  the  target  and  the  surround  and  on  the  extent  of  display  enhancement 
(if  pertinent).  As  the  resemblance  of  the  target  to  other  items  increases, 
processing  takes  longer. 

The  question  of  how  the  operator  uses  displays  in  performing 
search  and  detection  tasks  is  not  well  understood.  Is  the  pattern  of  search 
predictable  or  is  the  search  conducted  randomly  in  saccadic  fixations? 
Whether  the  approach  is  random  or  predictable,  is  one  decidedly  better 
than  the  other?  Can  more  efficient  strategies  for  search  be  taught?  These 
and  similar  data  are  required  regarding  search  behavior. 

In  addition  to  detecting  and  identifying  signals  , perceptual 
tasks  require  man  to  select  certain  aspects  of  the  signal  environment  or 
determine  patterns  in  the  visual  presentation.  Pattern  recognition  plays 
an  important  part  in  complex  skill  performance  since  it  serves  to  reduce 
the  number  of  different  stimuli  to  which  man  must  attend. 
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3 o 4. 2.2  Amount  and  Distribution  of  Input  Information.  Considerable 
research  has  been  accomplished  on  the  effects  of  speed  and  load  variables 
on  performance.  Of  concern  here  are  the  number  of  input  sources  to 
which  the  man  must  attend  and  the  frequency  and  rate  of  event  occurrence. 
Miller  (1956)  deduced  that  man  is  capable  of  processing  about  3 bits  of 
information  for  a discrete  stimulus  event.  Based  on  a number  of  labora- 
tory situations,  the  human,  perceptually,  is  able  to  identify  absolutely 
from  six  to  ten  steps  along  any  physical  dimension  (Howell  and  Brigg^, 

1959;  Alluisi,  1957). 

In  situations  where  a number  of  similar  stimuli  are  presented 
simultaneously  (e.g. , a "dot11  pattern  of  target  blips  on  a radar  display), 
evidence  indicates  that  more  than  one  perceptual  process  is  involved  in 
judging  the  number  of  targets  simultaneously  presented.  Man  is  able  to 
subitize  from  six  to  eight  blips  . Beyond  this  number,  man  estimates  the 
amount,  and  this  is  manifested  by  an  increase  in  the  slope  of  the  perfor- 
mance decrement  when  plotted  against  the  number  of  blips  presented  (i.e.  , 
amount  of  information  in  the  total  stimulus  group)  (Kaufman,  et  al,  1949). 

Experiments  on  the  span  of  attention  (number  of  items  perceived 
simultaneously)  have  utilized  dots  to  determine  span  limits.  Typically, 
attention  span  is  about  eight  dots  (reported  at  50%  accuracy)  (Woodworth, 
1938).  This  span,  however,  depends  on  exposure  time  of  the  signals. 
Averbach  ( 1963)  found  that  only  one  item  can  be  reported  for  40  millisecond 
exposures  with  level  of  accuracy  increasing  by  one  item  for  every  increase 
of  10  milliseconds  until  the  span  of  eight  items  is  reached.  This  suggests 
that  limitations  in  the  span  of  attention  depend  upon  the  limits  of  memory. 
What  are  the  limits  of  man's  ability  to  retain  information  presented  a 
single  time?  About  8 items  is  the  average  memory  span  (based  on  tradi- 
tional memory  span  expe  iments  employing  college  students).  With  prac- 
tice this  can  be  increased  to  about  twelve  items.  Memory  span  also  varies 
with  the  type  of  item  the  individual  has  to  report  (Fitts  and  Posner,  1967). 

The  spatial-temporal  aspects  of  information  processing  is  also 
of  importance  since  in  many  instances  man  must  respond  to  events  that 
are  spaced  in  the  time  dimension.  Hick's  (1952)  findings  suggest  the 
general  conclusion  that  response  time  varies  as  a logarithmic  function  of 
the  number  of  stimulus  events  but  the  rate  of  gain  of  information  is  constant. 
When  the  man  is  required  to  respond  at  a faster  rate  or  at  the  same  rate 
but  to  a greater  number  of  stimuli,  performance  accuracy  will  decrease 
proportionately  so  that  the  same  amount  of  information  is  transmitted. 

Human  performance  is  limited  under  conditions  of  excessive 
temporal  input  demands  (e.g.  , multi-channel  events  , rapid  sequences  of 
events,  overlapping  signals).  The  researches  by  Conrad  and  by  Mackworth 
(summarized  by  Howell  and  Briggs,  1959)  indicate  that  perceptual  failures 
accompanying  increases  in  speed  and  load  are  related  to:  a)  the  intervals 
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separating  stimuli  from  the  immediately  following  stimuli  as  well  as  from 
the  preceding  responses,  b)  the  bunching  of  short  inter  signal  intervals, 
c)  the  disorder  of  responding  associated  with  short  inter  signal  intervals, 
and  d)  the  stress  associated  with  increases  in  task  demands.  Performance 
also  suffers  proportionally  as  the  degree  of  overlap  among  signals  is  in- 
creased or  the  number  of  short  intervals  separating  them  from  each  other 
or  from  other  responses  is  increased. 

Based  on  the  literature,  Howell  and  Briggs  (1959)  have  pro- 
vided the  following  recommendations. 

• Displays  utilizing  single  physical  dimensions  should  not 
present  more  than  8 or  10  alternative  signals  for  absolute 
identification  by  the  operator.  In  general,  adding  new 
dimensions  is  more  satisfactory  than  adding  steps  along 
a single  dimension,  although  the  contribution  of  each  new 
dimension  is  likely  to  decrease  as  the  number  of  com- 
pounded dimensions  is  increased.  The  greater  the  number 
of  alternative  stimuli  to  which  the  operator  must  respond, 
and  the  greater  his  uncertainty  regarding  which  of  the 
alternatives  will  appear  at  a given  time,  the  greater  will 
be  the  latency  of  his  response.  If  forced  to  operate  at  a 
constant  speed,  accuracy  will  suffer  as  stimulus  uncer- 
tainty is  increased.  The  specific  capacity  values  for  human 
information  processing  cannot  be  given  directly  since  a 
number  of  additional  variables  enter  into  such  a determina- 
tion, including  the  adequacy  of  the  coding  scheme,  the 
compatibility  of  the  stimulus  and  response,  the  severity 
of  distractions  (e.g.  , noise),  the  degree  of  time-sharing 
requirements  5 and  the  number  of  responses  required. 

« Techniques  for  improving  infor mation-handling  capabilitie s 
include:  a)  the  addition  of  redundancy  to  the  signals 

(particularly  when  the  situation  is  "noisy11  or  the  signals 
are  not  easily  distinguishable),  b)  the  weighting  of  signals 
according  to  importance  and  relative  probability  of  occur- 
rence (informing  the  operator  of  these  weights  to  reduce 
his  uncertainty)  , c)  the  physical  arrangement  of  signals 
to  take  advantage  of  the  above  weightings  as  well  as  any 
sequencing  effects  which  might  reduce  the  operator’s 
uncertainty  regarding  signal  occurrence,  d)  the  utilization 
of  S-R.  compatibility  stereotypes  in  selecting  controls  for 
specific  displays  , e)  the  use  of  the  auditory  modality  when 
considerable  time-sharing  is  involved. 
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. In  tasks  which  demand  the  precise  timing  of  responses  to 
continuous  inputs,  considerable  attention  should  be  paid 
to  the  sequencing  of  events  . The  closer  a signal  is  to  a 
response,  the  more  apt  this  signal  is  to  be  missed.  Hence, 
raising  signal  speed  will  cause  an  increase  in  frequency 
of  missed  signals  , although  the  temporal  accuracy  of  each 
response  will  be  minimally  affected.  Raising  the  speed 
will  also  tend  to  bring  about  a predominance  of  late  re- 
sponses. Increasing  the  number  of  displays  over  which 
signals  are  presented  is  likely  to  create  a stressful  situa- 
tion similar  in  effect  to  that  resulting  from  an  overall  speed 
increase.  It  has  been  suggested  that  optimal  performance 
can  be  achieved  by  displaying  only  those  signals  which  are 
to  be  responded  to  currently  and  in  the  near  future  (the 
so-called  MnowM  type  of  display).  Other  suggestions  include, 
allowing  the  operator  to  regulate  overall  pace  of  inputs,  or 
where  this  is  not  possible,  setting  speed  approximately  to 
a normal  distribution  of  inter  stimulus  intervals;  and  re- 
ducing the  number  of  extremely  short  inter  stimulus  intervals 

When  the  operator  must  choose  an  information  source  from 
among  a group  of  such  sources,  his  decision  is  based  heavily  on  subjective 
probability  estimates  and  priority  values.  Howell  and  Briggs  (1959)  sug- 
gest that  the  operator  usually  responds  to  the  highest  combination  of 
probability  and  payoff  values  in  preference  to  either  higher  probability  or 
payoff  values  alone,  and  will  choose  high  probability  values  when  the  com- 
bination is  equated.  (Other  task  and  organismic  variables  may  also,  how- 
ever, enter  into  the  decision.) 

In  decision-making  situations  where  the  quality  of  the  decision 
takes  priority  over  speed,  models  have  been  proposed  based  on  game  theor 
and  statistical  decision  theory  which  form  bases  for  the  formulation  of 
rational  decisions  under  uncertainty  (Edwards  , 1954).  Pew  (1965)  has 
written: 
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MIn  a game  against  nature  the  possible  states  of 
nature  (indicated  by  index  i;  i = 1,  2,  . . - , n) 

and  the  available  courses  of  action  (indicated  by 
index  j;  j = 1,  2,  . ..  , m)  form  a two-dimensional 

decision  matrix  defining  the  outcome  structure 
of  the  game.  To  each  cell  of  this  matrix  is 
assigned  a value  Vj^j,  the  amount  to  be  gained  or 
lost,  associated  with  each  outcome.  A further 
relevant  variable  is  the  probability  P^  associated 
with  each  of  the  alternative  states  of  nature.  This 
probability  reflects  the  likelihood  that  any  particular 
state  is,  in  fact,  the  true  state  at  the  time  the 
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decision  is  to  be  made.  It  is  possible  to  propose 
many  different  criteria  of  good  decisions  based  on 
the  information  given.  Not  all  will  lead  to  tbe 
same  decision.  Maximizing  expected  value  (EV) 
is  one  criterion  which  is  appropriate  to  many 
situations  . Under  this  criterion  the  operator 
selects  the  action  (j)  so  as  to  maximize  over  m 
the  function  n thus  taking  into 

EV  = £ VijPi 
i=l 

account  both  the  probability  of  each  possible  state 
and  the  values  associated  with  all  pos s ible  out- 
comes . M (pg.  31-13) 


This  appears  to  be  a good  guide  to  follow  in  making  rational  decisions 
under  uncertainty.  It  is,  however,  not  descriptive  of  what  man  actually 
does  . The  logic  of  maximizing  expected  value  is  sound,  but  rarely  are 
objective  data  about  or  V^j  available  . Since  each  decision  situation  is 
unique  and  we  see  only  a single  outcome  revealed,  it  is  often  difficult  to 
obtain  the  classical  kind  of  relative-frequency  assessment  of  I?i . Simi- 
larly, only  in  certain  limited  circumstances  can  a well-defined  value  be 
placed  on  an  outcome.  Research  has  shown  that  man  deviates  significantly 
from  expected  value  predictions,  and  occasionally  fails  to  satisfy  even  the 
simplest  axioms  of  rational  behavior.  To  circumvent  the  limitations  of 
expected- value  maximization  as  a description  of  actual  decision-making 
behavior,  the  objective -probability  should  be  replaced  with  a perceived 
probability  and  the  actual  value  replaced  with  the  perceived  equivalent  of 
value,  utility.  !,Subjective  probability  and  utility  usually  are  assumed  to 
bear  a monotonic  relation  to  their  respective  objective  quantities.  The 
resultant  theory  is  called  subjectively  expected  utility  maximization  (SEU), 
and  it  has  been  shown  to  be  capable  of  describing  decision-making  behavior 
over  a wide  range  of  conditions.  Its  limitation  lies  in  the  looseness  of  the 
description  inherent  in  a model  with  two  parameters  which  are  only  in- 
directly measurable  and  difficult  to  infer  independently.  The  SEU  model 
has  been  applied  mainly  to  static  decision-making  tasks,  the  cases  in 
which  all  the  evidence  that  is  to  be  available  arrives  at  once  and  a single 
decision  is  required.  In  contrast  with  this  type  of  task  is  the  more  general 
case  of  dynamic  decision  making  in  which  the  operator  processes  informa- 
tion as  it  arrives  and  makes  a sequence  of  one  or  more  decisions  which 
may  or  may  not  be  interdependent.  While  this  description  is  more  typical 
of  the  real  world,  it  is  also  more  difficult  to  analyze.  The  dynamic-pro- 
graming methods  are  relevant  here , as  is  Bayesian  statistical  inference.  n 
(Pew,  1965;  pg.  31-14.) 

3.4. 2. 3 Distortions  in  Input  Information.  The  trainee  must  often,  as  a 
matter  of  coarse,  respond  to  distortions  in  the  input  information  since  the s 
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occur  in  the  operational  system.  Since  various  forms  of  distortion  tend 
to  degrade  the  information  processing  capabilities,  the  relationships  are 
of  importance  to  design  particularly  for  establishing  levels  of  task  realism 
and  for  deliberately  graduating  the  level  of  task  difficulty  in  simulation  to 
achieve  defined  training  objectives.  Several  forms  of  distortion  relevant 
to  human  factors  design  are  described  here  addressed  primarily  to  visual 
display  requirements. 

3. 4. 2. 3.1  The  Influence  of  Noise.  A number  of  investigations  have 

consistently  revealed  that  signal  identification  capability  in  static  displays 
is  degraded  as  noise  levels  are  increased. . In  these  instances  the  noise 
serves  to  increase  task  difficulty  both  in  terms  of  clutter  (signal  and  noise) 
and  ambiguity  in  signal  identification,  and,  for  dynamic  displays,  in  terms 
of  relevant/irrelevant  signals.  In  form  discrimination  experiments  con- 
cerned with  sorting  metric  figures  (Rappaport,  1957),  the  effects  of  re- 
dundancy on  the  recognition  of  visual  patterns  in  visual  noise  were  examined 
When  noise  elements  were  present,  rate  of  sorting  was  degraded.  How- 
ever, when  redundancy  was  added  under  the  background  noise  condition, 
rapid  discrimination  was  facilitated.  There  was  also  the  suggestion  that 
redundancy  may  be  detrimental  when  introduced  in  too  great  amounts  or  in 
an  ineffective  way. 


French  (1954)  studied  the  effect  of  randomly  scattered  points 
of  light  (visual  noise)  on  the  recognition  of  random  dot  patterns  of  varying 
complexity  (pattern  dots  varied  from  two  to  nine;  noise  dots  from  one  to 
eight).  The  subject  was  required  to  identify  each  test  pattern  within  5 
seconds  after  presentation  using  a noise-free  figure  as  the  sample.  In- 
creasing the  complexity  of  the  target  pattern  (i.e.  , increasing  the  number 
of  elements)  improved  recognition  performance  progressively.  However, 
increasing  the  complexity  of  the  visual  nois e produced  a progre  s sive  decre- 
ment in  target  recognition.  Increasing  the  elements  in  the  target  pattern 
beyond  eight  does  not  appear  to  influence  further  improvement  in  perfor- 
mance. In  general,  recognition  performance  improved  as  the  ratio  of 
number  of  target  dots  to  noise  dots  increased  up  to  a target- to-nois e ratio 
of  3:1. 


For  compensatory  tracking  tasks  , the  addition  of  increasing 
amounts  of  noise  to  the  input  signal  reduces  performance  significantly  (a 
roughly  linear  function  of  RMS  error  relative  to  noise  level).  Studies  by 
Kahn  and  Mazina  (1957)  required  the  trainee  to  track  a coherent  course 
signal  to  which  was  added  the  noise  signal.  Error  was  greater  at  all 
noise  levels  as  the  input  signal  was  raised  from  0.9  to  4.  5 cpm,  Howell 
and  Briggs  (1959b)  found  similar  increases  in  tracking  error  resulting 
from  increases'  in  visual  noise.  Noise  was  added  selectively  to  the  input, 
the  feedback,  the  input  plus  feedback  or  the  error  signals  on  the  compen- 
satory tracking  display.  The  trainee  was  not  able  to  filter  out  the  noise 
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since  his  error,  when  he  was  forced  to  filter  the  noise,  was  greater  than 
that  possibly  achieved  by  tracking  perfectly  the  noise  signal. 

It  is  interesting  to  note  that  in  a transfer  of  training  study 
utilizing  a compensating  tracking  task  simulating  an  F-8  6 aircraft  E series 
fire  control  system  (Briggs,  Eitts,  and  Bahrick,  1957)?  visual  noise  de- 
graded performance  at  all  stages  of  learning.  On  the  transfer  trials, 
however,  all  groups  were  found  to  be  of  comparable  proficiency.  Several 
training  procedures  did  not  differentially  affect  tracking  performance 
under  varied  conditions  of  noise.  The  authors  concluded  that  visual  noise 
depresses  performance  levels  but  has  little  net  effect  upon  learning  in  this 
skill  task. 

Howell  and  Briggs  (1959)  provide  the  following  summary 
based  on  laboratory  researches.  .h 

• Noise,  or  unwanted  random  signals,  in  the  input 
has  been  found  to  degrade  performance  seriously 
on  a number  of  different  types  of  perceptual  task: 
detection  of  auditory  signals,  detection  of  visual 
signals,  recognition  of  matrix  dot  patterns  or 
similar  nonsense  forms,  and  tracking  under  both 
compensatory  and  pur  suit  mode  s . 

• The  variable  of  prime  importance  in  the  considera- 
tion of  noise  distortion  is  the  ratio  of  signal-to-noise 
magnitude . Hence,  failing  to  lower  the  noise  level, 
the  same  result  can  be  achieved  by  raising  the  signal 
level.  In  auditory  displays  this  is  a matter  of  am- 
plifying signal  energy  as  in  the  case  of  certain  visual 
displays,  e.g.,  intensity  of  radar  returns  and  ampli- 
tude of  signal  excursions  in  a tracking  task.  In  other 
visual  situations  the  analogous  operation  is  increas- 
ing the  number  of  signal  elements.  In  dot  matrices, 
the  signal- to-noise  ratio  or  the  number  of  dots  de- 
fining the  pattern  relative  to  the  number  of  random 
or  "noise"  dots  appears  to  be  the  most  significant 
variable.  For  a given  number  of  signal  dots, 
errors  increase  as  a negatively  accelerated 
function  of  noise  . Similar  findings  obtain  with 
respect  to  metric  figures  in  that  figure  redun- 
dancy is  effective  in  combating  noise  decrements  . 

. Adding  redundancy  in  auditory  as  well  as  visual 

displays  is  a particularly  effective  means  of  combat- 
ing noise  effects  . When  it  is  possible  to  reduce 
distortion  on  some,  but  not  all,  display  channels. 
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it  appears  tliat  the  greatest  benefit  will  result  from  op- 
timizing those  displaying  input  information.  Tlie  dis- 
tortion of  feedback  information  is  less  serious  owing  to 
its  inherent  redundancy:  proprioceptive  as  well  as  visual 

cues  are  available  in  most  control  tasks. 


• For  dynamic  displays,  increasing  the  amount  of  noise 
appearing  on  a tracking  display  brings  about  a nearly 
linear  decrease  in  accuracy  of  performance  (at  least 
for  relatively  low  noise  levels).  In  this  situation 
noise  appears  as  a random  perturbation  of  the  coherent 
movement  patterns  described  by  the  display  elements. 
Introduction  of  noise  s elec tivelyN  to  the  various  display 
elements  (i.e.  , input  signal,  feedback  signal,  both 
input  and  feedback  signals,  and  the  error  signal  in 
a compensatory  display)  reveals  that  performance 
is  seriously  affected  only  when  input  information  is 
involved.  It  is  presumed  that  when  visual  feedback 
information  is  disrupted,  the  operator  places  more 
reliance  on  proprioceptive  feedback  cues. 

3.4. 2. 3. 2 Signal  Inte  rmittency  . Signal  or  display  intermittance  is  a 

design  issue  in  situations  involving  discretely  sampled  information  such 
as  with  radar  displays  or  involving  time  sharing  in  multi- display  contexts. 
The  general  finding  from  the  literature  is  that  performance  decreases  as 
a function  of  the  display  inte rmittence  period  (Howell  and  Briggs,  1959). 

In  multi-display  situations,  man  sets  up  a scanning  pattern  and  his  perfor- 
mance is  slowed  due  to  attention  switching  and  eye  movement  times. 
Broadbent  (1955)  found  attention  switching  time  to  be  on  the  order  of  0 . 2 
seconds . 


Intermittency  degrades  tracking  performance;  as  the  percent 
time  of  viewing  the  signal  decreases,  performance  decreases  in  an  approx- 
imately linear  fashion  (Battig , et  al , 1955).  However,  with  simple  inputs 
and  no  requirement  for  quick  and  precise  response,  the  harmful  effects 
are  minimized.  The  effect  of  intermittency  is  modified  by  input  complex- 
ity, display  brightness,  and  the  frequency  of  inte  rmittence  . Flash  duration 
in  an  intermittent  signal  usually  affects  target  brightness,  with  brightness 
lessening  as  flash  durations  decrease.  For  example,  a brightness  of 
+ 10  ml  yields  better  performance  than  a brightness  of  0.05  ml  which  in 
turn  results  in  better  performance  than  a brightness  of  0 . 005  ml  (Voss, 
1955). 


A study  by  Poulton  (1957)  provided  for  selective  intermit- 
tency in  the  target  signal,  feedback,  and  target  and  feedback  combined  for 
a pur  suit  tracking  display  . On-time  was  0.4  second  duration  and  off- time 
intervals  varied  from  0.2  to  4.7  seconds.  In  all  instances  performance 
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was  degraded  (mean  error  score)  as  the  duration  of  off-periods  lengthened. 
Feedback  intermittence  was  least  disruptive  of  performance.  Several 
means  for  reducing  the  harmful  effects  (especially  during  early  stages  of 
training)  of  intermittency  have  been  summarized  by  Ely,  Bowen  and 
Orlansky  (1957). 


Anticipatory  information  should  be  provided 
by  the  display. 


• Brightness  level  of  the  display  should  be  kept  high. 

• In  multi-display  formats  requiring  the  development 
of  a scanning  pattern  by  the  operator  , displays 
should  be  designed  and  arranged  to  minimize 
viewing  time  and  attention -shift  time. 


• The  duration  of  each  intermittent  signal  should 

be  as  long  as  possible  and  the  rate  of  presentation 
should  be  as  fast  as  possible. 

» Aiding  or  quickening  should  be  used,  when 
applicable. 


In  general,  the  function  obtained  between  proficiency  meas- 
ures and  the  frequency  of  signal  glimpses  is  negatively  accelerated.  This 
relationship  has  been  observed  for  intermittent  feedback  as  well  as  for 
intermittent  input  information.  The  feedback  case  is  considerably  less 
detrimental  to  performance  than  is  the  input  case  and  this  suggests  further 
evidence  that  man  is  able  to  substitute  proprioceptive  for  visual  feedback 
information  when  the  latter  is  degraded  in  some  way  (Howell  and  Briggs, 
1959). 


3. 4.  2. 3.3  Time  Lags.  Lags  are  an  integral  part  of  many  control 

tasks  and  hence  must  be  considered  in  specifying  display  and  control  re- 
quirements. The  distinction  has  been  .made  between  display  and  control 
lags,  the  former  occurring  between  the  output  and  the  display  and  the 
latter  between  the  control  and  the  output.  Garvey , Sweeney  and  Birmingham 
( 1958)  employing  a compensatory  tracking  task  found  that  performance 
error  increased  linearly  as  display  lag  increased  to  0.75s  ec onds . Under 
control  lag  conditions,  error  leveled  off  after  0.20  seconds. 

Lags  can  be  characterized  in  terms  of  where  they  appear 
in  the  system.  The  three  major  types  are  transmission,  exponential  and 
sigmoid  lags.  Transmission  lags,  even  when  extremely  small  (e.g . , 

0.06  seconds)  , tend  to  degrade  performance  with  compensatory  displays. 

For  transmission  lags  in  a compensatory  task,  Warrick  (1949)  found  that 
performance  decrement  is  a linear  function  of  lag  from  0 to  320  milliseconds 


O 

ERIC 


~ 

■V*  ^ 


- 310  - 


NAVTRADEVCEN  69-C-0298-1 


for  exponential  lags,  Warrick  (1955)  obtained  an  approximately  linear  func- 
tion between  tirr.e -off- target  and  lags  from  0 to  1000  milliseconds  „ Ex- 
ponential and  sigmoid  lags  can  either  improve  or  degrade  performance 
depending  on  their  interactions  with  other  aspects  of  machine  dynamics 
(Ely,  Bowen  and  Orlansky , 1957),.  For  example,  Rockway  (1954)  found 
that  when  gain  is  optimally  set,  an  exponential  lag  added  between  control 
output  and  machine  output  will  degrade  performance  , If,  however,  the 
gain  is  set  too  high  (causing  continual  overshooting),  the  lag  will  serve  to 
reduce  output  amplitude  and  thus  improve  performance. 


Howell  and  Briggs  (1959)  hold  that  the  distortions  created 
by  system  lags  can  be  averted  in  many  complex  systems  by  displaying  to 
the  operator  feedback  informatic  a not  only  of  system  output  but  also  of  its 
derivatives.  This  technique,  known  as  ’’quickening,  n can  reduce  the  effects 
of  such  delays  on  performance  and  can,  in  addition,  greatly  simplify  the 
operator’s  task  (see  3.4. 3.2). 


In  summary,  distortions  of  input  information  must  be  reck- 
oned with  in  design,  not  only  in  terms  of  the  negative  features  which  in  cer- 
tain instances  must  be  minimized,  but  in  terms  of  the  positive  values  for 
training.  For  example,  noise  and  inte rmittency  distortions  are  of  some 
value  in  providing  a means  for  the  development  of  a training  strategy  and 
must  be  considered  as  a design  option  for  certain  task  situations.  The 
prominent  feature  is  that  task  complexity  can  be  increased  via  these  means 
and  graduated  levels  of  task  difficulty  may  be  specified  precisely. 

3.4. 2.4  Stimulus -Response  Compatibility.  The  human  does  not  have  a 
fixed  limit  for  the  rate  at  which  information  is  transmitted.  Processing 
rates  vary  as  a function  of  the  matching  of  stimulus  and  response  coding 
and  of  different  amounts  of  learning.  Fitts  (19  58)  has  summarized  a 
number  of  studies  in  which  s timulus -response  compatibility  is  a significant 
task  characteristic,  i.e.  , certain  stimulus  codes  are  responded  to  more 
accurately  and  more  rapidly  with  certain  response  cues. 

A number  of  studies  have  varied  the  assignment  of  responses 
to  stimuli.  Morin  and  Grant  (1955)  employed  reaction  time  as  a measure 
for  three  different  S-R  codes;  Fitts  and  Deininger  (1954)  also  employed 
reaction  time  as  a measure  for  various  stimulus  codes  and  for  several 
different  ways  of  assigning  stimuli  to  responses;  Fitts  and  Seeger  ( 19  5 3) 
used  measures  of  reaction  times,  errors  and  information-transmission 
indices  to  compare  nine  different  S-R  ensembles;  Garvey  and  Knowles 
(1954)  compared  performance  for  six  S-R  ensembles;  Garvey  and  Mitnick 
(1955)  examined  the  spatial  anchoring  effects  important  in  S-R  compati- 
bility. The  above  studies  , representative  of  the  literature  , have  yielded 
the  following:  ; 


323 


- 311 


NAVTRADEVCEN  69  -C  -0298  - 1 


• Performance  depends  both,  upon  the  stimulus  and  the  re- 
sponse codes.  A particular  response  code  may  yield 
greater  performance  with  one  stimulus  code  than  with 
another  . 

• Factors  contributing  to  the  s timulus  - res  pons  e com- 
patibility effect  are  population  stereotypes,  spatial 
anchoring  effects  , and  the  relative  probability  of 
occurrence  of  specific  stimuli. 

« Performance,  measured  by  speed  and  accuracy  is 
usually  in  agreement  with  the  compatibility  of  the 
S-R  code  in  terms  of  population  stereotypes  . 

• The  effects  of  compatibility  will  continue  over  long 
periods  of  practice . 


• When  an  ordered  set  of  stimuli  is  used,  reversal 
of  stimuli  yields  better  performance  than  does  a 
random  assignment  of  stimuli  to  responses.  An 
incompatible  S-R  code  while  slower  than  a com- 
patible one  seems  to  be  effective  provided  some 
well-learned  rule  is  supplied. 

• The  source  of  compatibility  effects  appear  to 
reside  in  previous  learning  and  in  genetic  bases. 


Fitts  (1954)  has  stated  that  because  all  motor  responses  involve 
a distribution  of  force  in  time  and  space,  the  spatial  characteristics  of 
the  stimulus  code  is  especially  important  in  determining  S-R  compatibility 
where  rapid  motor  responses  are  required.  Compatibility  effects  are 
interpreted  in  terms  of  probability  learning,  the  adequacy  of  man’s  ex- 
pectancies relative  to  the  set  of  alternatives  present  in  a particular  task 
situation,  and  the  degree  of  re-encoding  of  information  required  in  the 
S-R  s equence . , . 


S-R  compatibility  has  also  been  examined  irr  tracking  tasks. 
Howell  and  Briggs  ( 1959V  have  summarized  the  research  concerned  with 
such  issues  as  control  movements  relative  to  displayed  stimulus  positions  , 
the  effects  of  display  variables  in  complex  tracking  tasks  ana  the  congru- 
ence of  S-R  sets  in  pictorial  reference  indicators.  For  example,  fly-from 
or  inside-out  displays  are  superior  to  fly -'to  or  outside-in  displays  in  the 
pictorial  representation  of  position.  For  circular  display-control  con- 
figurations, similar  display  and  control  movement  is  generally  superior 
to  reversed  rotation,  although  with  the  null  position  at  6 o’clock  the  extent 
of  this  superiority  is  doubtful.  Regarding  the  optimum  null  position,  the 
9 o’clock  position  has  been  found  to  be  particularly  advantageous  when  two 


eric: 


<rvCT*  * 


Sep*:. 


-312  - 


NA  V TRADE  VC  EN  69-C-0298-1 


separate  compensatory  displays  are  used  and  these  are  aligned  horizontally 
When  two  such  displays  are  aligned  vertically,  however,  the  12  o'clock 
position  becomes  superior  . The  12  o 'clock  position  also  appears  to  be 
superior  when  one  display  is  used.  These  writers  make  the  following 
recommendations  for  design:  ; 

. Control- display  combinations  should  always  be  selected 
to  take  advantage  of  S-R  compatibility  effects/  Many  such 
preferred  relationships  have  already  been  determined. 

Very  generally,  spatial  correspondence  of  multiple 
stimulus  and  r esponse  elements  ( e . g . ■,  - position  of  lights 
and  response  buttons  in  a matrix)  appears  to  offer  a high 
degree  of  compatibility.  In  the  same  manner,  directional 
correspondence  of  moving  elements  (e.g.  , a left-moving 
cursor  on  a CRT  and  the  leftward  movement  of  a control 
column)  are  highly  compatible  . Verbal  responses  have 
also  been  found  to  be  more  compatible  than  manual 
responses  (i.e.  , key  pressing)  in  relation  to  alpha- 
numeric stimuli.  For  unique  situations  , data  should  be 
collected  to  determine  the  stereotypes  of  the  potential 
operator  population  with  regard  to  preferred  display- 
control  relationships.  These  expressed  preferences 
should  then  be  tested  using  models  of  the  alternative 
combinations.  Generally  speaking,  the  differences  in 
preference  and  performance  are  marked,  so  that  rela- 
tively few  data  would  be  required  . 

9 Several  additional  principles  may  be  found  useful  in  r v 
optimizing  compatibility  effects.  When  large  groups 
of  stimuli  and  accompanying  response  elements  are 
involved  (as  is  the  case  in  matrices,  columns,  etc.), 
spatial  reference  lines  can  be  used  to  "break  up"  the 
groups  and  thereby  improve  performance.  In  addition, 
assigning  high -probability  events  to  spatial  anchor 
points  (i.e.,  edges,  tops  , bottoms  , etc.)  may  improve 
overall  tr  a ns  mis  sion  efficiency  . 

3.4.2. 5 Vigilance.  How  man  performs  in  situations  characterized  by 

infrequent  event  occurrence  over  time  (weak,  relatively  rare  signals)  is 
also  of  interest  to  design,  particularly  when  he  functions  as  a monitor  in 
various  system  situations  . Prominent  among  these  minimum  input  situa- 
tions for  training  design  are  the  sonar  and  radar  vigils  , monitoring  of 
system  states  (in  and  out -of-  tolerance!)  under  various  modes  of  operation 
(e  . g.  , semi-automatic)  , and;  monitoring  multi- display  situations  of  vary- 
ing complexities  and  of  varying  information  feedback  capabilities  . 
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A substantial  body  of  literature  exists  on  vigilance  behavior 
and  performance  under  low  frequency  input  conditions  and  a considerable 
array  of  research  findings  have  been  assembled  for  a variety  of  task 
situations  and  methods.  It  is  suggested  that  the  reader  consult  the  reports 
prepared  by  Buckner  and  Me Grath  ( 1 963)  , Human  Factor  s Research  (1968), 
Jerison  and  Pickett  (1963),  Swets  and  Kr istofferson  (1970),  Broadbent 
(1958)  and  Frankmann  and  Adams  (196/i)  for  a sampling  of  the  issues, 
problems  and  findings.  Theory  development  accounting  for  vigilance  and 
monitoring  behavior  has  developed  similarly.  Prominent  among  these 
are  the  Filter  Theory  (Broadbent,  1958),  Arousal  Theory  (Scott,  1957), 
Reinforcement  Theory  (Jerison  and  Pickett,  1963),  and  Expectancy  Hypo- 
thesis (Deese,  1955),  as  well  as  other  related  positions  or  hypotheses. 

Our  purpose  is  to  extract  from  this  significant  body  of  litera- 
ture, concepts  and  information  pertinent  to  the  design  of  displays  for 
training  systems. 

The  research  literature  on  human  vigilance  indicates  a heavy 
involvement  in  studies  concerned  with  man’s  ability  to  detect  signals  pre- 
sented in  simple  displays.  An  example  of  this  is  suggested  in  Figure  38 
which  illustrates  the  commonly  achieved  decrement  function  in  human 
vigilance.  As  an  oversimplification,  the  classic  finding  is  a performance 
decrement  showing  a monotonic  decline  in  signal  detection  as  a function  of 
length  of  vigil. 

The  major  theories  would  predict  the  classical  performance 
decrement  under  prolonged  vigil  conditions  (Buckner  and  McGrath,  1963). 
For  example,  Filter  Theory  holds  that  because  of  man’s  limited  percep- 
tual capacity,  a selective  operation  is  performed  on  all  inputs  to  the 
system.  During  the  monitoring  task,  the  man  will  select  irrelevant  com- 
peting stimuli  with  increasing  frequency  over  time.  The  result  is  a de- 
cline in  the  percentage  of  signals  detected.  For  Arousal  Theory,  the 
prediction  of  the  classical  vigilance  decrement  is  based  on  the  assvimption 
that  long-term  monotonous  stimulus  conditions  will  lead  to  a lowering  of 
cortical  arousal  and  thus  to  less  efficient  responding. 

However,  the  findings  are  mixed  regarding  a sensitivity  decre- 
ment for  all  kinds  of  vigilance  tasks.  Some  yield  a reduced  sensitivity 
over  time;  others  do  not  suffer  with  the  passage  of  time.  A prominent 
variable  to  account  for  this  is  the  type  of  display  employed  (Swets  and 
Kristoffer  son,  1970). 

Probability  of  detection  is  affected  also  by  temporal  signal 
uncertainty.  An  example  of  laboratory  results  relating  signal  rate  to 
probability  of  detection  is  presented  in  Figure  39.  The  differences  among 
the  curves  may  also  be  due  to  the  type  of  display  employed  and  variations 
in  procedure. 
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Figure  38. 


Three  performance  curves  from  different 
laboratories  illustrating  the  decrement  function 
in  human  vigilance.  A-Mackworth  (1950); 
B-Jerison  and  Wallis  (1957);  C-Jenkins  (1953). 
All  curves  based  on  signal  rates  of  about  30/hr 
and  display  with  no  spatial  uncertainty. 

(from  Jerison  and  Pickett,  1963) 


Figure  39 . 


Percent  signals  detected  as  a func  tion  of  log 
signal  frequency.  A-Deese  and  Ormond  (1953); 
B-*  Kappa  u£  and  Powe  (1959);  C-Jenkins  (1953, 
1958);  D- Jerison  { 1959)  . (The  log  scale,  used 
for  convenience  in  graphing,  has  no  theoretical 
significance  at  this  time.) 

{from  Jerison  and  Pickett,  1963) 
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The  vigilance  decrement,  characteristically  found  in  low 
signal  frequency,  simple  monitoring  tasks  does  not  seem  to  occur  in  the 
monitoring  of  complex  visual  tasks  (Adams  and. Boulter,  1964),  particu- 
larly as  the  level  of  display  complexity  (stimulus  density)  increases. 

The  consensus,  however,  is  that  the  probability  of  signal 
detection  in  monitoring  low-frequency  events  will  usually  show  a decre- 
ment over  work  time,  although  the  decrement  may  ;ot  occur  if  the  task 
involves  spatial  as  well  as  temporal  uncertainty  of  the  signal  (Jerison 
and  Pickett,  1963).  Additional  relevant  variables  include:  signal  char- 

acteristics (frequency,  regularity,  size,  intensity  and  spatial  distribution); 
the  level  of  extraneous  visual  and  auditory  noise;  the  schedule  of  rein- 
forcement for  observing  responses  (Holland,  1958);  and  the  design  of  the 
displays  (Swets  and  Kristoffersou,  1970). 

A number  of  techniques  may  be  used  to  increase  the  likelihood 
that  signals  will  be  detected  in  low-frequency  event  situations.  These 
include  the  following: 

• Increase  the  number  of  targets  that  must  be  detected. 

The  probability  of  detection  (in  terms  of  wanted  signals) 
increases  according  to  the  logarithm  of  the  number  of 
signals  per  hour  . 

• Insert  occasional  false  or  ‘'dummy”  targets. 

• Provide  information  about  when  and  where  a signal 
is  likely  to  occur  (position  on  the  display).  A re- 
duction in  unpredictability  in  space  is  desired,  e.g.  , 
monitoring  will  be  improved  when  several  spatially 
separate  signal  sources  are  combined  in  a centrally 
placed  display. 

• Increase  signal  probability  (increase  signals -per-unit- 
time  by  using  a longer  unit  of  time). 

• Introduce  unrelated  stimuli  at  various  intervals,  e.g.  , 
alerting  signals). 

• Intensify  or  enlarge  the  signals.  Display  size  should  be 
large  enough  to  be  legible  but  not  to  interfere  with  easy 
visual  search.  Thus,  a display  that  subtends  >5  degrees 
is  better  than  larger  versions  for  the  same  visual 
material. 

» Increase  the  absolute  detectability  of  the  signal  (increas- 
ing its  duration  improves  signal  to  noise  ratio). 
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# Provide  augmented  feedback  information  preferably 
via  training  equipment. 

. Add  redundancy  to  the  signals  (e.g.  , auditory  signals). 

. Introduce  artificial  conditions  in  the  displayed  situation. 

The  introduction  of  successive  events  at  regular  time 
intervals  (e.g.  , a signal  repeated  every  3-5  seconds 
until  detected)  will  assure  sustained  vigilance  under 
circumstances  that  otherwise  would  yield  a falling 
off  in  performance.- 

# Enhance  neglected  portions  of  a display  or  of  the  total 
displays  by  enhancing  the  stimulus  (e.g.  , differential 
brightening)  or  by  intensification  of  reference  markers 
on  the  display  in  order  to  distribute  the  man!s  attention, 

# Eye  movements  are  particularly  important  in  moni- 
toring PPI  screens.  Eor  example,  the  presence  of 

a scanning  radial  line  of  sight  has  been  shown  to  yield 
disproportionately  great  eye  fixations  in  a ring  at  the 
half-radius  region  of  the  circular  screen.  Thus,  de- 
tections are  most  likely  to  occur  at  a position  half-way 
out  from  the  center  of  a circular  display  (N . Mackworth, 
1960). 

It  should  be  noted  that  the  above  recommendations  are  based 
mostly  on  laboratory  data  of  which  there  is  a limited  amount  of  basic  data. 

In  essence,  the  results  obtained  from  the  vigilance  researches  support  the 
need  for  augmenting  trainee  compartment  displays  to  enhance  training 
effectiveness,  specifically  the  need  for  prompts,  cues,  knowledge  of  re- 
sults to  be  provided  in  equipment  as  well  as  deliberate  departures  from 
realism  with  respect  to  the  operational  world.  These  issues  will  be 
covered  subsequently  in  Chapter  3.7  of  this  Section. 

3.4.3  Design  Techniques  for  Enhancing  Displays  for  Training.  A 
number  of  techniques  are  available  for  the  design  of  displays  which  em- 
phasize instructional  efficiency.  These  may  be  grouped  under  the  rubric 
of  stimulus  coding  and  are  concerned  with  augmenting  the  identifiable  coding 
dimensions  by  which  information  can  best  be  displayed  to  the  man  to  achieve 
training  objectives  as  a function  of  level  of  learning.  The  techniques  pro- 
vide for  the  enhancing  of  signals  to  insure  detection  and  identification  of 
signals  in  cluttered  displays  and  also  to  insure  effective  monitoring  of 
signals  in  complex  displays  so  long  as  the  signal  is  of  concern  in  an  exer- 
cise (e.g.  , coding  of  targets  to  indicate  imminent  violation  of  minimum 
lateral  separation  in  an  air  traffic  control  display). 
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The  principle  underlying  this  usage  is  simple  - -provide  guidance 
to  essentially  delimit  the  frequency  and  severity  of  incorrect  responses 
in  performance  by  emphasizing  relative  aspects  of  training  and  limiting 
the  exploration  time  in  performance*  This  takes  on  two  aspects:  1)  the 

signals/targets  must  be  generated  and  coded  in  such  ways  as  to  insure 
the  achievement  of  the  training  objectives  for  the  training  system;  2)  where 
stimulus  supports  are  desired,  these  should  be  organized  in  ways  compati- 
ble with  achieving  the  desired  training  objectives.  The  effectiveness  of  this 
guidance  depends  on  the  amount  given  and  the  point  in  training  at  which  it 
is  introduced.  It  is  most  efficient  at  the  outset  of  learning  (initial  training) 
when  offered  in  discreet  doses.  When  given  in  large  amounts  at  inoppor- 
tune times,  it  may  actually  interfere  with  learning.  Thus,  proper  use 
dictates  that  display  enhancement  must  not  continue  to  the  point  where  the 
trainee  becomes  overdependent  on  outside  aid,  for  skill  acquisition  requires 
performance  without  assistance.  Response  cues  must  be  eventually  with- 
drawn so  that  the  trainee  responds  to  the  stimulus  desired  in  terminal 
performance.  (On  cue  withdrawal  or  stimulus  fading,  see  Skinner,  I960.) 

3*4.3. 1 Information  Coding.  Information  may  be  presented  in  a variety 
of  ways  to  the  performer.  Some  of  these  ways  are  more  effective  than 
others  in  enhancing  speed  and  accuracy  of  human  performance.  Thus,  the 
form  of  the  information  presentation  and  how  it  is  encoded  is  of  great  rele- 
vance to  design  for  training. 

Considerable  research  has  been  accomplished  on  coding  vari- 
ables , specifically,  on  such  commonly  employed  means  as:  color,  shape, 

size,  intensity,  orientation,  alpha-numerics,  line  length,  pulsing  signals 
and  the  use  of  target  traces  to  indicate  previous  position.  The  literature 
is  sizeable  and  the  reader  is  urged  to  consult  the  many  available  human 
engineering  compendiums  for  specific  information  (for  example,  two  such 
sources  are  the  reports  by  Fitts  (1954),  and  by  Baker  and  Grether  ( 1954))  . 
Some  generalized  findings  from  the  literature  are  outlined  below* 

* About  10-12  different  colors  is  the  maximum  that 
can  be  consistently  identified.  However,  in  terms 
of  current  color  generation  methods,  7 should  be 
considered  a practical  limit.  In  fact,  ,a  limit  of  4 

is  desirable/to  alleviate  color  registration  problems. 

■ . / * , . _ . _ ( 

* A maximum  of  four  size  levels  is  desirable  from 
the  standpoint  of  clutter  and  from  limitations  in 
character  generation. 


Shape  coding  is  constrained  by  the  ability  to  generate 
the  characters. 
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. Intensity  is  not  a desirable  dimension  since  average 

display  density  may  be  too  close  to  minimum  allowable 
to  permit  a dimmer  symbol.  Brightest  intensity  may 
be  used  as  a means  of  attention-getting. 

• Orientation  of  a vector  is  useful  for  indicating  heading, 
usually  a maximum  of  eight;  vector  length  is  useful  for 
representing  gross  speed  range  indication;  time -to  - go 
may  be  coded  as  a circular  vector. 

• Alpha-numeric s are  commonly  used  and  offer  unlimited 
possibilities.  They  are  usually  assigned  meanings  that 
can  be  easily  interpreted. 

A meaningful  example  of  the  use  of  alpha-numerics  as  a display 
variable  is  shown  in  research  conducted  on  Air  Traffic  Control.  ^ A display 
variable  found  to  have  a pronounced,  effect  on  system  efficiency  in  terminal 
area  air  traffic  control  operations  is  the  coding  of  aircraft  blips  (Schipper , 
et  al,  1957).  A visual  code  was  developed  for  the  primary  display  as  a 
means  of  providing  controllers  with  knowledge  of  the  identity  of  incoming 
aircraft.  It  was  in  the  form  of  an  alpha-numeric  clock-code.  The  longer 
hand  of  the  clock  code  is  read  as  Alpha , Bravo , Charlie  or  Delta;  the 
shorter  hand  is  read  as  the  corresponding  clock  numeral.  Thus, 

1/  /| 

Alpha  1 Charlie  8 Delta  11 

The  code  finally  selected  resulted  from  research  on  coding  for  efficient 
handling  of  information.  The  procedure  included  scaling  the  stimulus  by 
means  of  absolute  information  transmitted  and  determining  the  rate  at 
which  this  information  was  transmitted  (Muller,  et  al,  1955).  The  effect 
of  the  use  of  this  identity  code  on  accumulated  delay  in  aircraft  landing  is 
shown  in  Figure  40.  As  the  rate  of  entry  (number  of  aircraft  entering  the 
terminal  area  per  unit  of  time)  is  increased  beyond  some  point,  measures 
of  system  efficiency  declined  uniformly.  The  average  control  time  showed 
a marked  upward  trend  as  the  average  time  separation  between  successive 
entries  decreased  beyond  one  aircraft  every  90  seconds  for  the  condition 
of  no  identity,  and  as  it  decreased  beyond  one  entry  every  75  seconds  for 
the  condition  in  which  identity  was  given.  The  number  of  conflicts  also 


An  extensive  research  program  was  conducted  at  the  Laboratory  of 
Aviation  Psychology,  Ohio  State  University  by  Paul  IsA . Fitts.  The  work, 
begun  in  1950,  was  supported  by  the  National  Research  Council  with  funds 
provided  by  the  Air  Navigation  Development  Board.  After  1952,  the  re- 
search program  was  supported  by  the  Wright  Air  Development  Center  , . 

Wright-Patterson  AFB  , Ohio  (Contracts  AH  3f3(^  1 6)  -43  and  AF  33(616)-361' 

erjk:  33l  ! 
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Figure  40.  S ys tern  Performance  at  Four  Entry  Hates 
Under  Identity  and  No -Identity  Conditions. 
ID  Indicates  Identity;  X-IDj  No  Identity, 
(from  S chipper  3 Kr af  t , Smode  and  Fitts  3 
1 957.) 
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increased  as  a function  of  increased  entry  rate.  These  data  are  for  a 
system  in  which  a single  controller  was  required  to  handle  all  aircraft, 
and  entries  deviated  randomly  around  the  mean  temporal  separation  and 
appeared  anywhere  within  a 90-degree  sector  of  the  periphery.  All  air- 
craft were  inbound,  and  had  to  be  moved  through  a 50 -mile  zone  in  order 
to  reach  the  GCA  gate. 


Another  technique  of  coding  may  be  described  as  target  en- 
hancement, which  involves  modification  of  one  or  more  of  the  stimulus 
dimensions  defining  the  object(s)  of  interest  in  the  display.  This  method 
is  particularly  amenable  to  computer  generated  synthetic  displays,  and  is 
normally  employed  as  a means  of  bringing  the  trainee's  attention  to  an 
object  (e.g.  , aircraft,  ground  target),  or  to  an  out-of- tolerance  condition 
(e.g.  , minimum  lateral  separation,  critical  weapon  release  distance) 
which  demands  an  immediate  and  appropriate  response  in  the  operational 
environment. 

The  most  common  techniques  comprising  this  form  of  coding 
involve  increasing  the  brightness  contrast  between  an  object  and  its  back- 
ground, changing  the  color  of  a target,  increasing  the  size  of  an  object,  or 
varying  any  one  of  these  dimensions  on  a temporal  basis  (e.g.  , flicker, 
undulation  of  object  size,  color  flashing  or  displaying  the  previous  X 
seconds  of  a moving  target  as  a means  of  enhancing  the  ability  to  detect 
slow-moving  targets).  These  latter  techniques  involving  some  sort  of 
temporal  variation  are  preferred  to  the  other  steady  — s (.ate  techniques 
mentioned  above.  For  all  practical  purposes,  these  techniques  are  con- 
sidered equally  effective  in  terms  of  this  attention-getting  power  and  have 
been  successfully  employed  in  a number  of  applications. 


Selection  of  a specific  technique  must  be  made  in  light  of  the 
display  medium  employed  and  the  ease  with  which  it  can  be  implemented 
in  the  system.  In  some  instances,  the  medium  dictates  the  techniques 
that  may  be  used  within  realistic  tost  limitations.  CRT  displays  are, 
for  example,  more  amenable  to  any  one  of  these  techniques  than  are  opti- 
cally  projected  displays. 

The  attention  - getting  power,  or  conspicuity  of  ’'enhanced  objects 
in  the  display  depend  on  the  rate  with  which  the  signal  is  pulsed.  The  ex- 
perimental literature  does  not  suggest  any  one  rate  as  being  optimum,  and 
the  various  optimum  rates  that  are  reported  vary  as  a function  of  viewing 
conditions,  light-dark  ratio,  and  task  loading.  The  consensus  of  these 
studies  (e.g.  5 Projector,  1957;  Gallup  and  Hambacher , 1956),  however, 
indicates  that  rates  of  2 to  4 per  second  define  the  optimum  zone  over  a 
wide  range  of  conditions.  It  is  recommended  that  these  rates  be  employed 
when  this  form  of  coding  is  considered  desirable. 
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3.4, 3,2  [Prediction  of  Input  Information.  Information  acquisition  and 

processing  can  be  simplified  as  a result  of  the  design  of  the  information 
interface  between  the  man  and  the  system.  This  concerns  the  technique 
of  providing  predictive  information  to  the  operator.  [Proficiency  in  a 
number  of  tasks  (e. g.,  docking  maneuvers,  carrier  landing,  maintaining 
lateral  and  vertical  separation  of  aircraft  in  air  traffic  control,  etc.) 
depends  on  man’s  ability  to  predict  the  immediate  future  path  of  a target 
or  of  own  vehicle  in  relation  to  a target. 

Howell  and  Briggs  (1959)  in  a review  of  studies  concerned  with 
man’s  ability  to  predict  motion,  to  predict  intersect  or  collision  courses, 
and  to  predict  future  position  of  moving  objects,  concluded  the  following: 

• Man  relies  primarily  on  position,  magnitude  and 
direction  characteristics  of  input  signals  in  extra- 
polating the  course  of  a moving  target.  Man  should 
not  be  required  to  estimate  the  acceleration  char- 
acteristics or  higher  derivatives  of  input  signals. 

. The  accuracy  of  linear  extrapolation  is  generally 

found  to  decrease  with  distance  extrapolated,  hence 
operators  should  be  provided  with  objective  methods 
for  extrapolating  long  distances.  What  constitutes 
a long  distance  is  dependent,  of  course,  upon  the 
size  of  the  display,  the  tolerable  error,  the  speed 
of  the  target,  and  a number  of  other  variables. 

At  a normal  reading  distance,  however,  a 1-inch 
extrapolation  will  result  in  an  average  of  0.2-inch 
error;  a 6-inch  extrapolation  in  an  average  of 
1.4-inch  error.  In  extrapolating  curved  courses 
such  as  the  point  at  which  an  aircraft  appearing 
on  a PPI  will  “roll  out”  of  a turn,  judgments 
become  poorer  as  the  radius  of  arc  is  increased 
(particularly  for  sharp  turn  angles).  For  a radius 
of  arc  of  3.03  mm.  , for  example,  variable  error 
has  been  found  to  be  1.50  mm.  and  constant  error 
to  be  0.97  mm;  for  a radius  of  24.24  mm.  , variable 
error  approaches  4.69  mm.  and  constant  error 
+7 .88. 

. The  extent  to  which  speed  of  target  movement  in- 
fluences extrapolation  and  collision  judgments  is 
not  clear.  In  some  cases,  raising  speed  appears 
to  degrade  pe r fo r man c e , and  in  other  s it  has  little 
or  no  effect.  It  is  fairly  certain,  however,  that  in 
projecting  the  course  of  targets  moving  at  unequal 
speeds  greater  error  results  than  for  targets  moving 
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at  equal  speeds.  Therefore,  any  objective  means 
of  improving  projections  of  targets  moving  at  un- 
equal speeds  can  be  expected  to  aid  performance 
and  should,  if  possible,  be  adopted. 


Several  other  variables  have  importance  for  specific 
types  of  display  situations.  Trails  of  constant  bright- 
ness, for  example,  should  be  used  to  denote  speed  for 
targets  approaching  each  other  at  an  oblique  angle; 
fading  trails  are  superior  v/hen  the  approach  forms  an 
acute  angle.  Eor  judgments  of  the  future  range  of 
moving  targets,  the  linear  PPI  display  is  superior  to 
the  exponential  FFI  or  B-scan  display.  Bearing  esti- 
mations are  particularly  poor  on  the  exponential  F3?I 
display. 


. It  does  not  appear  to  be  necessary  to  provide  man 

with  an  abundance  of  past  course  information  in  many 
types  of  perceptual  prediction  or  extrapolation  tasks. 

In  extrapolation,  for  example,  judgments  based  on  the 
minimum  number  of  points  necessary  to  define  a 
course  are  as  good  as  judgments  based  on  a row  of 
closely  arranged  points.  In  tracking  tasks,  predic- 
tions of  future  position  (and  therefore  error  scores) 
are  relatively  unaffected  by  the  display  of  past  course 
information.  Attention . should,  therefore,  be  directed 
toward  providing  the  operator  with  accurate  and  readily 
visible  recent  information  rather  than  an  extensive 
amount  of  past  information.  In  tracking  it  is  particularly 
important  that  the  operator  have  some  knowledge  of  the 
overall  statistical  properties  of  the  input  to  aid  in  his 
prediction  of  the  future. 

Displays  have  been  developed  which  obviate  the  requirement 
for  learning  the  dynamics  of  complex  control  systems.  One  of  these 
display  aids  is  the  ’’Quickened11  display,  a modification  of  a closed-loop 
manual  control  system  which  changes  the  task  performed  from  complex 
control  system  tracking  to  the  tracking  of  a positional  control  system 
without  lag.  The  display  shows  the  operator  how  to  make  control  move- 
ments proportional  to  the  size  of  the  displayed  error.  It  informs  him  as 
to  what  responses  should  be  made  to  bring  about  appropriate  system  out- 
puts . This  greatly  reduces  the  output  error  of  the  system  (Birmingham 
and  Taylor,  1954). 

A "predictor M display  provides  the  operator  a display  of  the 
response  characteristic  of  the  system  in  terms  of  predicted  system  out- 
put at  some  time  in  the  future.  This  type  of  display  simplifies  the  task 
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by  providing  the  operator  information  relative  to  the  future  of  the  variable 
being  controlled.  The  operator  is  not  required  to  integrate  and  extrapolate 
system  parameters  to  develop  a predictive  model  of  the  system  (Kelley, 
1960). 

These  types  of  displays  should  reduce  the  difficulty  of  tracking 
with  a complex  control  system  and  should  also  reduce  the  size  of  the  error 
of  the  control-system  output. 


3.4. 3. 3 Actual  vs.  Simulated  Targets.  Another  of  the  issues  facing  the 
human  factors  specialist  concerns  the  design  decision  on  selecting  simulated 
versus  actual  targets  for  display.  This  selection  must  be  based  on  a careful 
consideration  of  the  visual  tasks  the  trainee  is  expected  to  learn  and  on  the 
conditions  under  which  they  are  performed  in  the  operational  environment. 


If  the  task  involves  detection,  recognition,  and  identification  of 
near  threshold  targets  in  complex  or  ’’high  noise”  environments,  then  actual 
targets  are  desirable.  However,  if  the  task  involves  tracking  or  maneuver- 
ing relative  to  the  target,  then  actual  targets  contribute  little  to  development 
of  this  skill.  In  this  case,  simulated  targets  would  serve  the  training  ob- 
jective quite  satisfactorily.  It  should  be  emphasized,  however,  that  track- 
ing ability  is  sometimes  heavily  influenced  by  target  fading  and  the  ability  of 
the  observer  to  reaquire  the  target  under  adverse  environmental  conditions. 
Under  these  circumstances,  tracking  performance  depends  quite  heavily  on 
the  ability  of  the  observer  to  accomplish  the  basic  visual  processes  of  de- 
tection, recognition,  and  identification,  even  though  they  may  occur  almost 
instantaneously.  For  this  reason,  actual  targets  must  sometimes  be  em- 
ployed in  simulator  displays  which  are  used  exclusively  for  developing 
tracking  skill. 

Mackie  and  Harabedian  (1964)  recommend  the  following  con- 
cerning simulated  vs.  actual  recorded  sonar  target  signals. 


O 

ERLC 


• For  training  in  target  detection,  the  use  of  variable 
signals  in  a background  of  noise  appears  mandatory. 

It  is  probable  that  highly  realistic  synthetic  signals 
represent  a better  choice  than  recorded  target  signals 
for  this  purpose  because  of  the  much  greater  stimulus 
and  problem  control  that  such  signals  afford. 

„ For  training  in  target  tracking,  the  use  of  less  realistic 
signals  can  be  tolerated.  However,  the  tracking  func- 
tion will  be  seriously  interfered  with,  whenever  vari- 
ations in  signal  intensity  or  excessive  masking  by 
noise  produce  temporary  loss  of  contact.  The  moment 
that  tracking  responses  become  dependent  upon  de- 
tection responses,  as  is  frequently  the  case  in  actual 
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operations  , the  signal  realism  requirements  for 
tracking  must  approach  those  recommended  for 
target  detection.  While  actual  target  recordings 
could  be  used  for  tracking  training,  synthetic 
signals  are  preferred. 

• For  training  in  active  classification,  actual  target 
recording  is  superior.  However,  the  use  of 
highly  realistic  synthetic  signals  appears  to  be 
entirely  feasible.  The  inclusion  of  variability  in 
all  displayed  signal  parameters  is  essential  for 
effective  training  in  active  classification.  The 
addition  of  background  (reverberation)  noise  is 
also  desirable.  And,  obviously,  all  major  informa- 
tion parameters  used  in  classification  must  be  repre- 
sented in  the  synthetic  signals  . 

• For  training  in  passive  classification,  recordings  of 
actual  target  signals  are  superior  to  synthetic  signals 
but  synthetic  signals  are  feasible.  The  inclusion  of 
variability  in  the  displayed  signal  parameters  also 

is  of  importance  in  developing  effective  synthetic  sig- 
nals for  passive  classification.  The  addition  of  much 
background  noise  is  of  questionable  value  , however, 
because  of  the  extreme  subtlety  of  some  of  the 
classification  clues. 

3.4.4  Research  Issues  . A number  of  research  issues  important  to  train- 
ing system  design  concerns  the  manner  in  which  man  copes  with  and  utilizes 
the  displayed  information  in  the  signal  environment.  Perhaps  the  most 
pervasive  issue  that  requires  resolution  centers  on  how  man  copes  with 
and  learns  to  discriminate  signals  in  a high  density  signal  environment. 
Evidence  indicates  that  man  extracts  only  large  elements  from  dynamic 
electronic  displays  and  ignores  other  features;  there  is  also  evidence  that 
man  develops  a set  for  what  he  is  looking  for  and  the  display  confirms  his 
expectation.  It  is  desirable  to  establish  what  elements  man  requires  and 
uses  in  complex  high-density  displays.;  For  example,  if  one  can  establish 
the  elements  the  man  requires  in  a specific  display  situation,  is  it  reason- 
able to  simulate  only  these  and  npaintri  in  the  background?  How  does  man 
utilize  chronologically  a scope  display  as  a function  of  task  requirements 
(e.g.  , land  mass  display,  terrain  avoidance  or  following , etc  . );  what  does 
he  attend  to  in  target  recognition,  in  target  identification?  Similarly,  in 
multi-display  contexts,  how  is  mode  selection  and  information  integration 
accomplished  to  determine  what  he  has  developing  at  any  given  time?  A 
pertinent  design  issue,  for  example,  is  do  you  present  all  the  information 
involved  in  a situation  or  do  you  simplify  the  presentation?  The  importance 
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of  cueing  requirements  is  implied  but  not  necessarily  at  the  amplitude  and 
frequency  found  in  the  operational  environment.  Similarly,  the  issue  of 
pattern  recognition  in  clutter  is  of  concern,  i.e.  , the  decision-making 
process  involving  a translation  or  mapping  from  a large  set  of  stimuli  or 
arrays  to  a much  smaller  set  of  events  or  names.  Thus,  the  complicated 
issue  of  display  interpretation  is  of  some  consequence  for  training  design. 
What  is  needed  is  a system  of  design  data  on  how  man  discriminates  among 
much  data  presented  simultaneously  or  in  rapid  sequence  for  major  classes 
of  task  situations,  and  how  he  combines  and  utilizes  information  from 
multi-display  sources  dissimilar  in  format  and  content. 

Another  issue  of  interest  to  design  is  the  extent  of  display  degra- 
dation. possible  without  performance  degradation,  e.g.  , how  much  reduction 
in  tolerances  is  permis  sible  as  a function  of  training  purpose?  Of  concern 
here  is  the  issue  of  abstract  vs.  concrete  representation  of  the  signal  en- 
vironment and  how  man  utilizes  the  information.  For  example,  does  the 
display  of  abstract  figures  and  events  on  a land  mass  display  (e.g.  , a river, 
a cluster  of  artifacts,  etc.),  instead  of  specific,  concrete  entities  (e.g., 
a named  river,  a named  shoreline,  etc.),  or  does  substituting  a mosaic 
which  simulates  a PPI  instead  of  higher  radar  resolution  on  the  PPI,  affect 
transfer  of  training  differentially?  The  understanding  of  this  issue  has 
important  consequences  for  the  design  of  complex  training  displays.  Allied 
to  the  above  is  the  value  of  ,,mediation,,  in  information  display.  A study  by 
Silver,  Jones  and  Landis  (19  65)  indicated  that  with  skillful  radar  observers, 
paper  and  pencil  format  displays  can  be  presented  as  successfully  as  live 
simulation  of  the  same  information.  For  novice  trainees,  however,  live 
simulation  is  desired  with  an  emphasis  on  the  relationships  between  stimuli 
rather  than  on  single  stimulus  variables. 

Also  desirable  is  the  development  of  a data  base  for  simulator 
design  on  the  number  of  s ignals / targets  that  the  man  can  react  to  and  pro- 
cess effectively  (i.e.,  displayed  simultaneously  or  in  fast  sequence)  for 
various  classes  of  tasks  and  display  characteristics,  and  for  target  char- 
acteristics (e.g.,  maneuverability,  speed,  patterning,  etc.).  It  is  not 
clear,  for  example,  where  the  performance  breakdown  points  are  as  to  the 
number  of  targets  that  a man  can  successfully  handle  in  his  span  of  control; 
it  is  not  clear  how  targets  are  best  generated  in  terms  of  man^  perceptual 
capabilities,  for  various  task/display  requirements  (e.g.,  dynamic  and 
slaved  target  relationships,  ganged  signals,  etc.). 
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3.5  MEASUREMENT  SYSTEM  DESIGN 

This  chapter  presents  concepts  and  data  pertinent  to  the  design  of  the 
performance  measurement  capability  in  training  devices.  Of  concern  are 
the  design  requirements  involved  in  providing  the  means  for  monitoring, 
scoring  and  evaluating  human  performance  in  training  systems.  Emphasis 
is  placed  on  automatic  human  performance  monitoring  and  recording  means 
(computer  programing  and  scoring  equipment)  for  providing  measures  and 
scores  that  can  be  organized,  manipulated  and  applied  to  achieve  the  pur- 
poses of  assessment.  The  design  issues  center  on  the  data  requirements 
for  deriving  measures,  the  selection  of  relevant  measures,  the  organiza- 
tion of  measurement  data,  the  display  of  scoring  information  (content  and 
technique),  and  the  recording  of  measurement  information. 

No  provision  is  made  in  this  subsection  to  present  human  performance 
measurement  technology  relevant  to  design.  The  principles  and  practices, 
the  pitfalls  and  weaknesses  in  defining  and  assessing  performance  is  a story 
in  itself  that  is  well  told  in  a congeries  of  documents  in  the  opep.  literature  . 
We  are  tacitly  assuming  the  involvement  of  this  primary  information  base 
in  decisions  on  selecting  from  the  available  design  alternatives.  In  addition, 
a procedure  involving  a sequence  of  steps  for  achieving  a.fheasurement 
capability  in  a training  device  has  been  explicated  in  Section  II  of  this  report. 
Thus,  we  suggest  that  the  reader  consult  the  procedure  for  developing  a 
measurement  system  and  examine  collaterally  the  body  of  literature  on  the 
available  measurement  technology.  As  an  aid,  a number  of  documents  are 
recommended  which  describe  in  an  authoritative  and  detailed  fashion,  im- 
portant issues  and  concerns  in  measuring  and  assessing  human  performance. 
These  books  and  reports  emphasize  key  topical  areas  in  measurement.  No 
single  document  is  sufficient  as  an  assist  in  designing  a measurement  sys- 
tem. The  specific  studies  deal  with  a range  of  considerations  involved  in 
the  development  of  a measurement  capability.  A representative  sampling 
from  among  many  excellent  sources  is  provided  below. 

General  treatises  on  proficiency  measurement 

Stevens,  S.S.  Mathematics  measurement  and  Psychophysics.  In 
S.S.  Stevens  (Ed.),  Handbook  of  Experimental  Psychology.  N.Y.: 
Wiley,  1951.  ~~~~~  — 

Glaser,  R.  and  Klaus,  D.  J.  Proficiency  measurement:  Assessing 

Human  Performance,  in  R.  Gagne  (Ed.),  Psychological  Principles 
in  Systems  Development.  N.Y.:  Holt,  Rinehart  and  Winston,  1963. 

Churchman,  C.W.  and  Ratoosh,  P.  (Eds.),  Measurement:  Defi- 

nitions  and  Theories.  N.Y.:  John  Wiley  and  Sons,  Inc.,  1959- 
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Obermayer,  R.W.  Simulation,  Models  and  Games:  Sources  of 

Measurement,  Human  Factors  Journal  6 (6),  December  1964, 
607-619.  _ 

Precision  and  Relevance  in  Proficiency  Measurement 


Thorndike,  R.  Personnel  Selection:  Test  and  Measurement 

Techniques,  JN  . Y . : Wiley,  1949. 

Lindquist,  E.L.  , et  al.  Educational  Measurement.  American 
Council  on  Education,  Washington,  D.C.,  1951. 

Joint  Committee  of  the  APA,  AERA  and  NCMUE.  Technical 
Recommendations  for  Psychological  Tests  and  Diagnostic 
Techniques.  Supplement  to  Psychological  Bulletin,  Vol.  51, 
1954. 

Statistical  Considerations 


Guilford,  J.P.  Psychometric  Methods,  2nd  Edition,  N.Y.: 
McGraw-Hill,  1954. 

Guilford,  J.P.  Fundamental  Statistics  in  Psychology  and 
Education,  N.Y.:  McGraw-Hill,  1956. 

Draper,  N.R.  and  Smith,  H.  Applied  Regression  Analysis, 

N.  Y. : John  Wiley  &c  Sons  , Inc  . , 1966. 

Torgerson,  W.S.  Theory  and  Methods  of  Scaling.  New  York: 
Wiley  and  Sons  , Inc  . , 1958  . 

Gulliksen,  H.  Paired  Comparisons  and  the  Logic  of  Measurement. 
Psychological  Review,  Vol.  53,  1946. 

Nunnally,  J.C.  Psychometric  Theory.  New  York:  McGraw-Hill, 

19  67.  ~ 

Criteria 


Obermayer,  R.W*  Measurement  Criteria  in  Man- Machine  Systems 
Simulation.  NASA  CR  257,  National  Aeronautics  and  Space  Admin- 
istration, Washington,  D.C.,  July  1965. 


Uhlaner,  J.E.  and  Drucker,  A.  J.  Criteria  for  Human  Perfor- 
mance .Resear  ch.  Human  Factors  Journal  6 (3),  June  1964,  265- 
278. 
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McCoy,  W.K.  Problems  of  Validity  of  Measures  in  Investigating 
Man- Machine  Systems.  Human  Factors,  19  63,  5,  37  3-377. 

Measurement  Applications  in  Military  Training  Contexts 


Many  studies  have  been  conducted,  sponsored  by  government  agencies. 
The  key  agencies  include  the  following: 


Civil  Aeronautics  Administration  (pre-WWII) 

USAAF  Aviation  Psychology  Program  (WWII) 

USAF  Human  Resources  Research  Center  (HRRC) 

USAF  Personnel  and  Training  Research  Center  (AFPTRC) 
Human  Resources  Research  Office  (HumRRO) 

National  Aeronautics  and  Space  Administration 
USAF  Aerospace  Medical  Research  Laboratories 
U . S . Naval  Training  Device  Center 
Federal  Aviation  Agency 


In  addition,  a number  of  specific  applied  studies  has  been  published 
which  deal  with  the  development,  application  and  review  of  measurement 
technology.  These  consider  a variety  of  specific  issues  related  to  the 
utilization  of  training  systems  , and  provide  valuable  background  informa- 
tion for  the  design  of  a measurement  capability  in  training  devices.  Repre- 
sentative studies  are  listed  below. 


Ericksen,  S.C.  A Review  of  the  Literature  on  Methods  of  Measuring 
Pilot  Proficiency.  Research  Bulletin,  52-25  Human  Resources 
Research  Center,  Lackland  AFB , Texas,  1952. 

Krumm , R.L.  & Farina,  A.J.  Effectiveness  of  Integrated  Flight 
Simulator  Training  in  Promoting  B-52  Crew  Coordination.  AMRL- 
TDR  62-1,  Aerospace  Medical  Research  Laboratories,  Wright- 
Patter  son  AFB  , Ohio,  1962. 

Greer,  G.D.,  Smith,  W.D.  and  Hatfield , J . L . Improving  Flight 
Proficiency  Evaluation  in  Army  Helicopter  Pilot  Training.  HumRRO 
TR  No.  77,  U.S.  Army  Aviation  Human  Research  Unit,  Ft.  Rucker, 
Alabama,  May  1962. 

Smode,  A.F.,  Gruber,  A.  and  Ely,  J.H.  The  Measurement  of 
Advanced  Flight  Vehicle  Crew  Proficiency  in  Synthetic  Ground 
Environments.  AMRL-TDR  62-2,  Aerospace  !Medical  P.esearch 
Laboratories,  Wright- Patter  son  AFB , Ohio,  February  1962. 

Buckhout,  R.  and  Cotterman,  T.E.  Considerations  intheDesign  of 
Automatic  Proficiency  Measurement  Equipment  in  Simulators, 
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3.5.1  Measurement  Capabilities  in  Current  Training  Devices  . A well- 
developed  measurement  capability  has  not  been  the  hallmark  of  current 
training  devices.  Training  systems  on-line  today  are  characterized  by 
instructor  monitoring,  control  and  evaluation  of  training  (manual  setup  of 
initial  conditions  and  enroute  modifications  of  the  training  exercise),  with 
minimal  assists  from  computer  scoring  of  performance  or  from  equipments 
designed  primarily  for  measurement.  Stated  simply,  simulators  designed 
for  training  are  not  utilized  in  any  substantial  way  for  scoring  individual 
or  team  performance.  They  are  not  equipped  intrinsically  with  well-con- 
ceived scoring  subsystems.  The  measurement  weakness  results  in  most 
cases  from  a design  approach  that  emphasizes  fidelity  of  simulation  with 
measurement  as  a by-product,  i.e.  , picking  off  and  outputing  what  can  be 
obtained,  once  the  synthetic  environment  has  been  created.  At  most, 
simple  time  to  perform  and  system  error  scores  are  available,  together 
with  communications  (audio)  records.  In  some  instances,  a visual  mission- 
situation  display  (panorama)  is  provided  for  use  in  reconstructing  the  exer- 
cise after  its  completion;  hard  copy  records  (computer  printout  of  time  and 
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event  happenings  during  the  exercise)  may  be  provided.  In  vehicle  simula- 
tors, provision  is  often  made  for  recording  hardware,  the  outputs  of  which 
are  employed  as  adjuncts  to  assessment,  for  example,  those  which  depict 
a time  history  of  an  event  or  a course  of  action  (flight  path  recorders  to 
show  a plot  of  ground  track,  approach  recorders  to  show  the  path  to  a radio 
station,  etc.). 

In  large  part,  the  measurement  capability  in  simulators,  particu- 
larly those  representing  surface  and  subsurface  vehicles,  has  achieved 
fruition  during  the  development  of  utilization  procedures,  once  the  device 
is  constructed  and  on-line.  Only  limited  provisions  have  been  made  during 
device  design  for  objective  scoring  and  recording  of  performance.  Also, 
measurement  has  served  most  prominently  in  providing  feedback  to  the 
trainee  about  his  performance.  The  verbal  post-mission  critique  based  on 
instructor  judgments  about  performance  usually  supported  by  a situational 
display  of  the  reconstructed  mission,  has  been  a prevailing  performance 
assessment  mode. 

Several  examples  of  such  after-the-fact  developments  of  objective 
scoring  and  recording  capabilities  are  summarized  next  to  indicate  the 
basic  subjectivity  and  lack  of  precision  in  performance  assessment. 
(Attempts  at  automatic  scoring  modes  are  discussed  separately  in  paragraph 
3.5.3.) 


A recent  example  is  Device  14A2,  ASROC/ ASW  Early  Attack 
Weapon  System  Trainer.  This  ASW  tactical  team  training  device  employs 
a system  involving  three  types  of  performance  observation  which  essen- 
tially organize  instructor  judgments  about  performance  (Dunlap  and 
Associates,  1969).  1 No  overall  scores  or  grades  are  developed  for  the 
team.  Observations  are  made  solely  to  determine  if  the  training  objectives 
are  met  for  each  training  exercise.  The  three  types  of  performance  ob- 
servation are  as  follows: 

« Checklists  are  employed  for  recording  the  occur rence  / non- 
occurrence of  critical  events  that  must  be  accomplished 
during  each  exercise  in  a graduated  training  sequence. 

• Quantitative  scores  are  obtained  on  time  and  accuracy  in 
the  team  development  of  fire  control  solutions  and  ASROC 
weapon  firings  . 

^The  measurement  capability  was  developed  with  the  current  device  con- 
figuration as  a given.  Thus,  performance  observation  was  the  method 
used  for  team  evaluation  in  Device  14A2. 
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. Instructor  judgments  are  assembled  primarily  on  evidence 
of  team  coordination  (threat  assessment,  employment  of 
the  platform,  sensors  and  weapons)  and  on  the  quality  of 
communications  (intra-  and  inter -ship)  focusing  on  content, 
on  anticipation  of  the  information  needs  of  other  members 
and  on  lateness  in  conveying  information. 

Thus,  only  relatively  simple  time  and  accuracy  readouts  are  provided  in 
the  device.  No  hard  copy  records  of  performance  are  available  for  devel- 
oping  normative  data  and  for  school  record-keeping. 


The  performance  standards  employed  center  on  correctness  of 
team  procedural  sequences  and  weapon  aim  point  errors  based  on  error 
curves.  The  curves  present  normative  data  obtained  via  computer  simu- 
lation of  sonar  operator  tracking  and  target  motion  analysis  at  the  Attack 
Director  console  (UB/plot).  The  data  used  in  establishing  the  standards 
are  derived  from  skilled  operator  performance  under  a variety  of  task  and 
mission  conditions. 


An  example  of  a submarine  team  trainer  is  Device  21A39-  This 
is  a simulated  FBM  submarine  attack  center  team  trainer  comprising  a 
Mk  113  Fire  Control  System  and  associated  sonar  plotting  gear.  The  pur- 
pose of  training  is  to  exercise  the  team  in  fire  control  solutions  and  attack 
procedures.  Since  periscope  simulation  is  not  provided,  techniques  of 
bearing-only  solution  are  used.  No  attempt  is  made  to  train  in  tactics  re- 
lating to  the  positioning  of  own-ship,  choice  of  weapons  or  choice  of  targets 
in  multi-target  environments.  The  index  of  team  performance  is  fire 
control  solution  accuracy.  Performance  is  self-diagnosed;  the  team  deter- 
mines how  well  it  performed  and  decides  on  what  the  next  exercise  will  be. 
There  is  no  formal  briefing  or  critique  of  performance.  A large  screen 
display  and  a hard  copy  computer  printout  are  used  for  problem  recon- 
struction. The  hard  copy  printout  provides  a minute-by-minute  account 
of  the  following  sixteen  parameters. 


Time  (in  minutes) 
Bearing  rate 
True  bearing 
Generated  bearing 
Actual  range 
PK  range 
Actual  course 
PK  course 


Actual  speed 
PK  speed 
Angle -on -the  -bow 
Own  course 
Own  speed 
Own  depth 
WG  run 
Weapon  steers 


This  record  provides  a fairly  precise  statement  of  .the  relative  geometry 
between  own  sub  and  the  target.  It  also  provides  a basis  for  determining 
errors  between  actual  and  team- generated  values  for  all  relevant  parame- 
ters. In  addition  to  computer  printouts,  the  device  has  available  an  audio 
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tape  recorder  which  can  provide  real-time  recordings  of  communications 
over  the  voice  circuits. 

The  measurement  capability  is  concerned  with  gross  team  eval- 
uation (Jeantheau,  1970).  The  contributions  of  individuals  to  the  team  per- 
formance are  not  isolated;  the  data  base  records  only  the  outcomes  of  team 
activity.  Tactical  considerations  are  not  accounted  for;  the  measures  are 
used  only  to  evaluate  the  team’s  ability  to  solve  the  fire  control  problem. 
The  measures  obtained  are  the  following: 


. Range  maintenance  (cumulative  percent  of  time  within 
range  error  brackets).  Percent  range  error  is  plotted 
against  percent  of  problem  time  that  error  is  within  the 
range  bracket.  Thus,  performance  can  be  evaluated 
against  any  selected  error  in  terms  of  the  amount  of 
time  the  team  maintained  range  within  that  error 
tolerance  . 

• Range  error  at  time  of  firing  (computed  at  the  minute 
prior  to  weapon  release). 

m Bearing  rate  maintenance  (computed  difference  between 
actual  bearing  rate  and  PK  bearing  rate).  Bearing  rate 
error  is  plotted  in  terms  of  absolute  error. 

. Bearing  rate  error  at  time  of  firing  (computed  at  the 
minute  prior  to  weapon  release). 

Another  example  of  a device  not  having  an  inherent  measurement 


capability  at  the  time  of  field  installation  is  Device  IBZrZ,  maneuvering 
tactics  trainer.  This  is  a team  trainer  which  provides  instruction  in  sur- 
face ship  tactics  and,  to  a limited  extent,  submarine  and  aircraft  operations 
as  these  affect  surface  ship  tactics.  Training  is  provided  in  the  exercise 
of  tactical  command,  communications,  relative  motion,  close  formation 
tactical  maneuvers  and  the  use  of  tactical  publications.  Visual  presentation 
is  achieved  through  a projection  system  controlled  from  16  independent 
control  stations  (mockup  ships’  bridges  representative  of  a 2,200-ton 
destroyer).  Each  mockup  contains  a helm  unit,  a PPI  scope,  a status 
board  and  a two-channel  communications  system.  Each  ’’bridge”  is  linked 
to  a projector  through  a computer-driven  mechanism  which  projects  an 
outline  of  a ship  (identified  by  a letter)  on  a 16-foot  x 16-foot  vertical 
screen.  Two  instructor  stations  are  provided.  Each  station  has  a console 
which  controls  and  monitors  eight  ships.  Control  is  limited  to  voice  com- 
munication and  the  ability  to  take  ranges  and  bearings  from  any  ship  in  the 
eight-ship  division.  One  of  the  consoles  is  capable  of  repositioning  ships 
while  the  other  contains  the  switchboard  for  the  sound-powered  telephone 
system.  A measurement  system  was  developed  for  the  purpose  of 
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evaluating  the  effectiveness  of  the  device  as  a training  tool  (Jeantheau  and 
Andersen,  1966),  Checklists  were  developed  for  the  four  trainee  positions 
to  enable  instructor  judgments  about  team  performance.  The  following 
measures  were  obtained  via  instructor  judgments. 


• Supervisory  and  leadership  capability 
. Action  taken  for  maneuvers  executed 
. Use  of  proper  orders  and  terminology 
. Interpretation  of  tactical  signal 
. Maintaining  station  in  formation 


# Maintaining  timely  and  accurate  formation  plot 
m Maneuvering  >>oard  solution 

RT  Talker/Radar  Operator 

c Use  of  correct  terminology 

# PPI  scope  operation 

# RT  voice  procedure  and  terminology 

• PPI  scope  operation 

* Circuit  discipline 
. Voice  circuit  log 

Helmsman 

• Voice  procedure  and  terminology 

o Operation  of  wheel  and  engine  order  telegraph 

Team  Performance 

• Ability  to  work  together 
. Adherence  to  protocols 

„ General  success  of  team  goals 


Note  that  in  Devices  1BZ-2  and  14A2,  no  measures  of  relative 
geometry  of  the  vehicles  in  an  engagement,  and  no  hard  copy  printouts  of 
performance  were  available.  Thus,  reliance  was  placed  on  instructor 
observations  about  performance.  In  Device  21A39,  a hard  copy  record 
of  relative  geometry  was  available  (via  design),  thus,  derived  measures 
of  performance  were  obtainable. 
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3.5.2  Research  Developing  a Measurement  Capability  for  Training,  As 
a prelude  to  the  subsequent  discussion  it  is  worthy  to  note  that  measure- 
ment technology  has  not  yet  developed  to  enable  the  unambiguous  assess- 
ment of  complex  human  performance  in  operational  systems.  Assuming 
that  unlimited  types  and  amounts  of  information  could  be  acquired,  the  basic 
questions  that  resist  systematic  solution  continue  to  be:  what  information  is 

necessary  or  required?  how  is  the  information  to  be  assembled  and  organ- 
ized? what  kinds  of  analyses  and  how  often  is  the  performance  to  be  exam- 
ined? A basic  difficulty  has  been  the  absence  of  adequate  operational  criteria. 
Due  to  the  absence  of  effective  measurement  guidelines  and  operational  cri- 
teria, the  opinion  of  experts  has  been  a primary  factor  in  the  selection  and 
organization  of  performance  measures.  (In  our  opinion,  it  is  this  funda- 
mental lack  in  measurement  technique  that  has  thwarted  the  design  emphasis 
on  automated  scoring  in  training  simulators.) 

A number  of  studies  have  attempted  to  structure  a measurement 
capability  for  use  in  various  operational  systems  contexts.  These  efforts, 
most  of  which  have  been  concerned  with  flying  training,  have  attempted  to 
improve,  in  objective,  standard  ways,  the  ability  to  assess  the  progress  and 
the  outcomes  of  a course  of  instruction.  The  results  of  these  studies  are  of 
presumable  interest  to  training  device  design  and  utilization  in  terms  of  the 
insights  provided  by  the  various  approaches  for  defining  measures  and  for 
implementing  scoring  procedures. 

A variety  of  studies  over  the  last  two  decades  have  reported  the 
development  of  objective  flight  checks.  In  essence,  these  flight  checks 
(checklist  format)  comprise  both  subjective  and  objective  measures  of  ele- 
ments of  the  pilots  job,  with  subjective  judgments  made  on  small,  well- 
defined  aspects  of  performance  (e.g.  , application  of  power  during  a maneu- 
ver). The  first  program  to  obtain  detailed  and  objective  inflight  proficiency 
measures  (begun  in  1939)  resulted  in  the  Ohio  State  Flight  Inventory 
(Ericksen,  1952).  Other  inflight  measurement  of  performance  attempts 
included  the  Army  Air  Force  World  War  II  grading  system  (Miller,  1947), 
the  Air  Force  Human  Resources  Research  Center  program  on  assessing 
student  pilots  (Smith,  Flexman  and  Houston,  1952);  the  CAA  Airline  Pilot 
flight  checks  (CAA,  1950);  the  FAA  flight  checks  for  airline  transport 
rating  (FAA  Form  342A)  and  the  airman  pr oficiency/qualification  check 
(FAA  Form  3111);  the  Army  Aviation  Pilot  performance  description  record 
(Greer,  Smith  and  Hatfield,  1962)  and  the  Air  Force  standardization/ eval- 
uation checks  (for  example,  SACM  51-4,  1966).  As  a group,  the  inflight 
performance  checks  have  proved  lacking  for  a number  of  reasons.  Less 
than  complete  information  is  provided  by  the  measures  and  methods. 

The  prevailing  approach  is  that  decisions  on  what  aspects  of  behavior  to 
sample,  and  when  and  under  what  conditions  observations  are  made,  are 
left  to  intuitive  judgment.  Measures  obtained  are  often  indeterminently 
associated  with  overall  proficiency.  In  many  instances,  measurement  is 
sufficiently  difficult  that  the  practice  is  to  ofetjain  what  is  me  a sure  able 
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rather  than  what  is  desired.  There  is  also  the  frequent  inability  to  detect 
and  assess  differences  in  performance  when  they,  in  fact,  exist.  The 
difficulty  continues  to  be  the  inability  to  structure  the  inflight  environment 
so  that  accuracy,  reliability  and  validity  of  measurement  are  within 
tolerances  . 

A program  currently  under  development  by  the  Training  Research 
Division,  Air  Force  Human  Resources  Laboratory*  is  attempting  to  build 
an  inflight  data  recording  system  and  a ground-based  automated  perfor- 
mance evaluation  capability  for  use  in  undergraduate  pilot  training.  A T-37 
aircraft  has  been  instrumented  to  record  2 3 flight  parameters  from  which 
an  additional  8 variables  are  computed.  Figure  41  describes  the  perfor- 
mance variables  selected.  The  barrel  roll  and  the  lazy-8  are  the  maneu- 
vers which  will  be  initially  analyzed. 

An  experimental  program  has  installed  audio/ video  recording 
equipment  in  a training  aircraft  as  a means  for  assessing  pilot  performance. 
This  program  (Neese,  1968;  Purifoy,  1968)  developed  methods  for  inte- 
grating an  airborne  audio/ video  recording  system  and  ground  playback 
equipment  into  portions  of  Air  Force  undergraduate  pilot  training.  The 
playback  equipment  included  a stock  video  recorder  and  an  18-inch,  screen 
black  and  white  TV  monitor;  stock  audio/ video  equipment  was  mounted  in 
T-37  and  T-38  aircraft.  This  method  of  recording  selected  pilot  perfor- 
mances served  as  an  adjunct  to  current  training  and  assessment  practices 
(i.e.  , no  alteration  of  each  instructor  pilot’s  instructional  approach)  and 
the  measures  of  performance  selected  were  common  with  the  standard 
training  program  to  permit  comparisons.  The  twenty-two  minutes  of  tape 
available  during  each  mission,  sampled  the  maneuvers  performed  (per 
mission  number)  in  the  ATC  flight  syllabus  for  the  aircraft.  Attempts 
are  being  made  to  correlate  these  records  with  instructor  judgment  ratings. 

3.  5.2.1  Studies  Concerned  with  Developing  an  Objective  Measurement 
Capability  in  Training  Devices.  Few  studies  are  reported  in  the  literature 
which  bear  directly  on  implementing  an  objective  measurement  capability 
in  simulators  for  assessing  training  performance.  During  the  1950s, 
attempts  at  objective  scoring  focused  on  means  for  recording  portions  of 
tasks  in  specific  skill  areas  of  the  pilot’s  job  with  the  recorded  data  anal- 
yzed at  a later  time.  Determining  the  feasibility  of  mechanical  scoring 
methods  for  simulators  was  then  the  vogue  (see  Smode , Hall  and  Meyer, 
1966,  for  a summary  of  these  efforts). 


*Air  Force  Human  Resources  Laboratory,  Training  Research  Division, 
Wright-Patterson  AFB,  Ohio. 
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To  be  sure,  various  proposals  have  been  made  for  developing 
complete  mo<ir.urement-assessment  packages  integral  to  the  synthetic 
ground  environment.  These  proposals  emphasize  logical  and  systematic 
development,  beginning  with  the  precise  determination  of  measurement 
objectives  and  culminating  in  an  integrated  behavior /measures/ scoring 
hardware  array  for  evaluating  trainee  performance  (see  Smode,  Gruber 
and  Ely,  1962;  Buckhout  and  Cotterman,  1963;  SAE , 1968).  However, 
implementation  of  such  and  similar  schemes  has  not  been  systematically 
pm  sued. 


Attempts  have  been  made  to  instrument  flight  simulators  for 
specific  measurement  purposes,  but  utilizing  existing  on-line  trainers. 

The  study  by  Bowen,  et  al  (1966),  cited  in  paragraph  3.5.3  is  an  example. 

A number  of  studies  have  been  concerned  with  analyzing  the 
guidance  and  control  performance  in  flight  simulations  employing  a variety 
of  measurement  approaches.  These  studies,  however,  were  not  structured 
for  training  purposes;  the  simulation  served  as  a test  method  for  deriving 
performance  information.  The  information  objectives  included  system 
feasibility  demonstrations;  comparisons  of  specific  components  of  the 
vehicle,  development  of  quantitative  models  of  man-machine  performance; 
vehicle  handling  qualities  investigations;  and  total  system  performance 
evaluation.  Representative  studies  within  each  of  these  test  methods  have 
been  analyzed  by  Obermayer  and  Muckier  (1964),  and  they  describe  the 
measurement  requirements  and  problems  and  the  differing  approaches  to 
appropriate  measurement  for  each  of  these  objectives.  In  essence,  the 
measurement  selection  involves  three  basic  categories: 

Vehicle  stability  measures  Response  characteristics  of  the 

vehicle  system 

Preference  measures  Subjective  measures  obtained  via 

questionnaire,  paired  comparisons, 
psychophysical  judgment,  or  seman- 
tic differential  (semantic  profile) 

Response  measures  Time  history  measures  and  ampli- 

tude distribution  measures  (time  on 
target,  RMS,  average  error,  aver- 
age absolute  error,  standard 
deviation) 

Obermayer  and  Muckier  (1964)  caution,  however,  that  despite  the  varieties 
of  measures  obtainable  for  the  above  purposes,  none  are  subject  to  unequi- 
vocal interpretation.  So  far  as  the  purposes  of  the  present  report  are 
concerned,  little  commonality  exists  between  measurement  for  training 
and  measurement  for  system  test  and  development.  The  approaches 


NAVTRADEVCEN  69-C-0298-1 


described  above,  however,  provide  the  reader  useful  information  to  aid  in 
selecting  from  among  available  design  alternatives. 

3.5.3  Automatic  Monitoring,  Evaluation  and  Scoring.  The  engineering 
feasibility  of  automated  scoring  has  been  heightened  with  advances  in  the 
design  of  digital  computers  employed  in  simulators,  and  the  current  trend 
in  design  is  towards  the  development  of  the  automated  monitoring,  evalua- 
tion and  scoring  capability  for  training  devices.  A number  of  studies  have 
been  published  concerned  with  demonstrating  the  feasibility  of  automatic 
scoring  and  with  developing  pertinent  design  requirements.  Some  are 
plainly  speculative,  emphasizing  only  requirements;  others  have  duplicated 
certain  aspects  of  instructional  requirements  in  simulators  (primarily  in 
the  flight  context)  in  order  to  demonstrate  feasibility  for  design. 

However,  this  automatic  measurement  capability  (computer  pro- 
graming and  display  and  recording  of  performance  information)  has  not 
been  effectively  implemented  for  pui  poses  of  training.  None  of  the  pub- 
lished research  has  been  concerned  with  actual  operational  trainer  imple- 
mentation. Studies  which  illustrate  the  emphasis  on  design  feasibility  and 
on  requirements  are  described  below. 

Curtis  s - Wright  C or  poration  (I3enenati,  Hull,  Korobow,  and 
Nienaltowski , 1962)  developed  a design  for  an  automatic  monitoring  system 
for  the  flight  simulator  utilizing  available  engineering  means  for  recording 
pertinent  mission  parameters.  Trainee  performance  was  scored  on  the 
basis  of  comparing  the  parameters  monitored  withthe  programed  perfor- 
mance standards,  e.g.  , errors  in  performance  were  printed  out  for  use 
by  an  instructor.  The  report  was  limited  to  a specification  of  principles 
involved  in  monitoring  and  scoring  selected  parameters  and  did  not  deal 
with  the  design  of  a proficiency  measurement  system. 

One  of  the  first  attempts  to  monitor  pilot  performance  in  the 
simulator  was  the  UDOFT  program  (Sylvania,  1963).  Using  a general- 
purpose  digital  computer,  this  program  demonstrated  the  feasibility  of 
automatic  performance  monitoring.  Several  jet  qualified  pilots  flew  the 
simulator  in  the  F-100A  configuration  from  takeoff  to  altitude  and  main- 
tained a holding  pattern.  Pilots  were  scored  on  the  ability  to  abort  on 
takeoff  and  on  airstarts  following  simulated  flameouts  . 

An  automatic  task  sequencing  technique  was  developed  in  an  Air 
Force  study  ( ICurtzberg,  1963)  which  presaged  a type  of  automatic  program- 
ing for  task  scheduling  in  simulators.  The  purpose  of  the  study  was  to  in- 
vestigate the  feasibility  of  automating  the  instructor  function  of  sequencing 
of  tasks  for  presentation  to  trainees  in  flight  simulators.  Algorithms  for 
task  sequencing  in  real  time  were  formulated  for  two  classes  of  application: 
training  of  students  for  flight  vehicle  operation  (operation  teaching  mode), 
and  training  for  development  of  tactics  skills  (tactics  teaching  mode). 
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Fourteen  inflight  emergencies  were  flow-diagramed  together  with  measures 
of  performance  and  ranking  of  alternative  responses  available  to  the  trainee. 
In  the  training  sequence,  tasks  were  automatically  selected  and  presented, 
performance  compared  with  established  criteria  and  results  recorded,  and 
new  task  selection  made  on  the  basis  of  the  trainee’s  previous  performance. 
The  logic  of  the  technique  suggested  the  possibility  of  automatic  redirection 
of  the  training  sequence  as  a function  of  the  trainee’s  performance  threshold 
of  the  moment  (i.e.  , adaptive  sequencing).  The  fourteen  in-flight  emer- 
gency tasks  and  the  number  of  variations  of  operations  for  which  scores 
are  given  are  shown  below. 


No.  of 

Task  Variations 


Engine  Inlet  Icing  5 

Instrument  Failure  Due  to  Pitot  Icing  6 

Surface  Icing  3 

Smoke  from  Air  Conditioner  8 

Smoke  from  Communications  Panel  6 

Smoke  and  Fumes  Elimination  6 

Fuselage  Fire  1 1 

Engine  Fire  While  Cruising  10 

Fuel  Dumping  7 

Emergency  Descent  9 

Bailout  4 

Engine  Relight  5 

Engine  Air  Start  13 

Runaway  Stabilizer  Trim  18 


Total  Variations  111 


/ 

/ 


An  example  of  the  scoring  chart  for  the  task  of  Engine  Relight  is  shown 
in  Figure  42. 


Another  example  of  this  trend  toward  automatic  monitoring  of  pilot 
performance  in  simulator  training,  is  the  program  under  development  at 
the  Human  Resources  Laboratory,  Wright-Patter son  Air  Force  Base.  The 
research  effort  is  centered  on  developing  a digital  computer  program  for 
automatically  monitoring  human  performance  in  the  training  simulator 
(Knoop,  1966).  The  first  study  in  this  research-oriented  automatic 
monitoring  program  (called  RAMP)  provided  an  experimental  tool  for 
investigating  the  criterion  problem  as  well  as  serving  as  an  automatic 
performance  monitoring  system.  In  developing  the  automatic  monitoring 
program,  it  is  assumed  that  no  quantitative  criteria  exist.  The  problem 
is  approached  as  a programing  task,  i.e.  , not  accumulating  data  and 
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Figure  42.  Engine  Relight  (5  variations) 
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processing  it  in  a predetermined  manner  but  rather  establishing  decision- 
making techniques  to  accommodate  highly  variable  information  on  human 
performance.  The  computer  program  is  designed  to  assist  in  the  analysis 
and  determination  of  performance  measures  and  performance  criteria,  and 
using  these  criteria,  automatically  monitors  human  performance.  Part  of 
the  utility  of  the  automatic  monitoring  program  will  be  in  the  evaluation  of 
criteria  for  flight  tasks.  These  criteria  need  not  be  exact  and  may  be 
easily  altered.  Of  RAMP,  Knoop  says,  initially  its  effectiveness  will 
depend  on  the  accuracy  and  detail  of  the  inputs  provided  by  the  user. 

In  essence,  the  automatic  monitoring  program  is  designed  to  re- 
ceive inputs  regarding  criteria,  establish  requisite  matrices  for  consoli- 
dating these  inputs,  and  then  monitor  the  trainee’s  performance  in  order  to 
test  the  criteria.  The  features  of  the  monitor  (implemented  by  dynamic 
programing)  include:  criteria  analysis;  task  sequencing  (automatic  redirec- 

tion of  the  training  sequence  to  provide  individualized  instruction);  and 
automatic  scoring.  The  intent  of  automatic  scoring  is  to  "free-up"  the 
instructor  so  that  he  may  perform  observations  of  activities  difficult  to 
automate.  The  technique  assumes  the  existence  of  a communication  chan- 
nel from  RAMP  to  the  trainee. 

A second  study  in  this  program  (Knoop,  1968)  was  concerned  with 
designing,  implementing  and  evaluating  a prototype  automatic  monitoring 
program  based  on  the  programing  capabilities  derived  in  the  previous  study. 
This  involves  the  development  of  a computer  program  for  automatic  per- 
formance monitoring  which  enables  the  implementation  of  the  performance 
criteria  and  the  performance  evaluation  techniques  that  are  currently 
known.  A feature  of  this  is  the  ability  to  estimate  and  change,  as  required, 
those  criteria  that  are  unknown,  and  allow  the  instructor  to  select  the  types 
of  performance  information  most  useful  to  him  in  the  instructional  role. 

This  is  an  important  design  characteristic  since  no  well-grounded  body  of 
data  exists  on  relevant  and  exact  performance  measures  nor  are  there 
accepted  standards  or  baselines  for  assessing  performance  (i.e.  , the 
criterion  problem).  Thus,  the  prototype  software  package  for  automatic 
performance  monitoring  emphasizes  easily  changed  performance  criteria 
and  the  addition  of  subroutines  for  experimentation  in  criteria  evaluation. 
The  program  was  applied  to  5 types  of  automatic  monitoring,  using  an 
orbital  reentry  vehicle  model.  Knoop  defines  each  of  these  applications 
as  relevant  to  flight  simulator  training.  The  automatic  monitoring  pro- 
gram was  found  to  be  applicable  also  to  hardware  diagnostics  and  trouble- 
shooting in  a simulator  training  facility.  Based  on  this  work,  Knoop  (1968) 
feels  that,  "we  are  now  in  a position  to  recommend,  with  reasonable  re- 
liability, experience,  and  foresight,  the  type  of  program,  monitoring  capa- 
bilities, and  the  computer  memory  requirements  best  suited  for  an  auto- 
matic monitoring  system  for  use  with  flight  simulator  trainers.  The 
state-of-the-art  does  not  now  and  perhaps  never  will  permit  us  to  specify, 
unequivocably , optimal  performance  criteria,  performance  measurement 
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techniques,  and  instructor  or  trainee  feedback  for  flight  missions  in  general. 
It  is  therefore  important  that  automatic  monitoring  and  scoring  programs 
be  designed  so  that  advantage  may  be  taken,  on  a continual  basis,  of 
knowledge  gained  through  operational  experience  with  the  monitoring 
system.  M (p.97).  Future  research  will  involve  implementing  this  proto- 
type monitoring  system  in  a flight  simulator  for  field  evaluation-  Research 
will  also  be  conducted  using  the  prototype  system,  augmented  by  appropri- 
ate subroutines  to  develop  methods  for  evaluating  and  altering  user-defined 
performance  measures,  tolerances  and  criteria  automatically  as  perfor- 
mance data  are  accumulated. 


to  automate  the  scoring  of  portions  of  the  pilot's  tasks  in  Navy  training. 

The  research  investigated  the  effects  of  two  training  treatments  on  pilot 
performance  in  a Navy  Operational  Flight  Trainer  (OFT)  simulating  the 
A-4  aircraft.  As  part  of  the  study  objective,  scoring  devices  and  proce- 
dures were  devised  (utilizing  the  on-line  OFT)  which  would  provide  a reli- 
able basis  for  assessing  pilot  proficiency.  Two  groups  of  ten  pilots  each, 
transitioning  to  the  A-4  aircraft,  were  given  three  simulator  training 
sessions  spread  over  a period  of  20  weeks.  The  control  group  of  pilots 
flew  conventional  flights;  for  the  same  flights,  the  experimental  group  re- 
ceived several  levels  of  augmented  feedback  based  on  the  scoriUg  procedures 
developed.  Knowledge  of  performance  information  was  given  bit  by  bit 
(error  was  identified  when  made),  by  numerical  grade  for  each  Emergency 
procedure  (series  of  tasks),  and  by  an  overall  summary  score  at  mission 
completion.  The  results  indicated  that  objective  scoring  information  given 
immediately  to  the  pilot  in  a useful  form,  heightened  performance  . The 
study,  however,  was  conducted  in  the  operational  environment  on  a "non- 
interference" basis;  hence,  problems  in  strict  experimental  control  were 
experienced.  Matching  of  subjects  in  the  two  groups  was  not  possible. 

Also,  the  three  OFT  sessions  were  spread  over  a period  of  20  ^eeks,  and 
the  authors  contend  that  so  much  else  was  going  on  in  the  training  course 
that  the  OFT  experience  tended  to  be  "minimized"  An  enhancement  effect 
for  the  experimental  group  cannot  be  discounted  either,  since  these  pilots 
may  have  been  "urged"  to  know  the  emergency  procedures  well-  The 
scoring  instrumentation  which  was  designed  and  added  to  the  OFT  was  as 
follows:  (1)  A panel  of  lights  which  displayed  the  sequence  of  events  per- 

formed during  emergency  procedures.  A buzzer  sounded  when  a sequence 
was  in  error.  The  displayed  sequence  of  events  was  manually  recorded 
on  a scoring  sheet.  (2)  Electric  stopclocks  were  programed  to  give  re- 
sponse times  and  total  time  in  completing  a procedure.  (3)  A ]3*ush  eight- 
channel  pen  recorder  scored  and  documented  discrete,  continuous,  and 
mixed  events.  Comments  made  by  the  trainee  or  the  instructor  were  also 
recorded. 

Three  sets  of  independent  scores  of  pilot  skill  in  the  simulator 
were  developed. 


A study  by  Bowen,  Bishop,  Promisel,  and  Robins  (196&)  attempted 
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Emergency  Procedures  Scores: 
S (sequence  score) 


P (proportion  score) 


B (binary  score) 

Time  Scores: 

RT  (response  time) 

TT  (total  time) 
Aircraft  Handling  Scores: 
C (control  score) 

Instructor  Ratings 
Self  (pilot)  Ratings 


Number  of  steps  accomplished  in  a 
procedural  task,  weighted  for  number 
and  difficulty  of  steps  completed  and 
not  completed. 

Proportion  of  steps  correctly  accom- 
plished to  the  total  number  of  steps 
required  (simplified  version  of  the 
S score) . 

Proportion  of  the  number  of  total 
procedures  correctly  accomplished. 


Time  to  complete  correctly  the  first 
step  in  a procedure. 

Time  to  complete  an  entire  procedure. 


Rating  from  polygraph  records  of 
control  effectiveness  over  aircraft 
and  engine  parameters. 

Proportion  of  satisfactory  procedures, 

Five-point  scale  for  rating  emer- 
gency procedures,  precision  flight, 
and  general  pilot  ability. 


The  authors  suggest  that  valid  simulator  measures  require  that 
the  trainee  be  exposed  to  a multiplicity  of  tasks  and  events  similar  to  rea'.l 
flight  conditions  and  that  the  difficulty  level  be  equivalent  to  the  more  diffi- 
cult aspects  of  real  flight.  Ir.  this  way,  the  pilot  will  perform  in  a pattern 
of  priority  as  in  actual  flight  (e.g.  , timesharing,  attention,  shifts,  antici- 
pation of  events,  etc.).  The  pilot  would  be  practicing  in  a realistic  way 
the  actual  skill  requirements  of  flight.  Means  were  recommended  for 
objectively  recording  the  following  data: 

• Procedural  sequences  and  computation  of  a score  which 
accounts  for  the  relative  difficulty  of  sequential  steps. 
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. Deviations  from  required  flight  and  engine  parameters. 

Such  data  may  be  inspected  for  radical  deviations  or 
used  in  a computational  program  as  part  of  a score  for 
manual  control. 

. Response  time  to  unexpected  situations. 

m Accuracy  of  precision  flight,  including  a composite  score. 

m Accuracy  of  navigation. 

Finally,  the  scoring  devices  should  possess  a high-speed  capability 
with  hard-copy  printout  in  near-real  time.  A computer-based  system  is 
preferred  because  of  the  need  for  updating  and  improving  the  various  scor- 
ing and  recording  programs. 

3. 5. 3.1  The  New  Breed.  The  planning  for  new  synthetic  training  sys- 
tems makes  provision  for  the  automatic  monitoring,  evaluation  and  scoring 
capability  when  relevant  to  the  purpose  of  the  training.  Two  examples  are 
provided  to  indicate  the  innovative  design  features  and  elements  of  the 
measurement  system. 

3. 5. 3.1.1  Synthetic  Flight  Training  System,  Device  2B24.  The  design 
of  this  helicopter  instrument  flight  trainer  is  based  on  automated  training 
concepts.  These  are  applied  to  decrease  the  number  of  instructors  needed, 
to  provide  individualized  instruction  in  a multi- station  trainer,  to  produce 

a standardized  product,  to  allow  proficiency  advancement  in  part,  through 
adaptive  sequencing  and  to  provide  objective  and  comprehensive  performance 
evaluation  (Hundt,  1969).  The  core  of  the  system  is  the  computer  program- 
ing which  drives  the  simulator  to  provide  preprogramed  scenarios  and 
which  provides  for  automatic  monitoring,  evaluation  and  scoring  of  per- 
formance, displayed  via  computer- generated  CRTs  with  hard-copy  records 
of  performance.  A summary  of  the  system  elements  and  a resume  of  the 
innovative  design  features  for  training  are  presented  below. 

The  simulator  operates  in  three  modes: 

. Automatic  mode--This  is  the  primary  mode  and 
includes  adaptive  sequencing. 

. Semi-automatic  mode- - The  instructional  burden  is 
greatest  in  this  mode,  since  training  strategies  are 
under  instructor  contr ol  based  on  assists  from  the 
automatic  monitoring,  evaluation  and  scoring 
subsystem . 
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• Checkride  mode  — This  is  the  standard  evaluation 
mission(s)  and  instructor  involvement  is  minimal. 


The  instructional  functions  include  the  following: 

a.  Briefing  and  demons  tration-  - This  requires  the  selection 
of  briefings  and  demonstration  of  flight  maneuvers  from  a tape  library. 

b.  Student  practice  monitoring  and  support- -The  selection 
of  practice  material  for  imparting  procedures,  knowledges  and  skills  to 
trainees  and  the  gathering  of  data  necessary  for  assessment  is  involved 
here;  a training  strategy  is  developed  for  each  student  to  achieve  a reason- 
able learning  rate. 


c.  Assessment — Computer  programs  provide  assistance  by 
displaying  out- of- tolerance  conditions,  and  initiating  problem  freezes  when 
conditions  warrant;  the  instructor  also  has  records  of  ground  track,  alti- 
tude, airspeed,  excesses  in  safe  operating  limits,  communications,  and  a 
display  of  the  last  five  minutes  of  performance;  the  instructor  monitors  the 
problem  displays,  monitors  the  error  information  displays  and  communi- 
cates with  students. 

d.  Guidance- -Based  on  the  problem  situational  display  and 

performance  error  display,  the  instructor  options  include:  verbal  guid- 

ance, problem  freeze  with  guidanc  e/ demons  tration  based  on  the  diagnosis 
of  error. 

e.  Debriefing -- This  includes  debriefing  of  a flight  segment 
(involving  such  assists  as  tape  records  of  voice  procedures,  record  of 
ground  track,  error  excesses,  and  other  indicants);  and  total  evaluation  at 
exercise  completion  (involving  performance  as  a whole,  error  measure- 
ments, and  scores  identified  via  hard-copy  printouts  and  plots). 

A number  of  instructional  assists  are  provided  to  unburden 
the  instructor  in  operating  the  simulator  and  to  better  enable  him  to  select 
options  conducive  to  effective  instruction.  Only  those  assists  which  repre- 
sent innovations  for  training  are  described  next;  their  availability,  of 
course,  varies  as  a function  of  the  training  mode  being  employed. 


o 

ERIC 


m. 


a.  Auxiliary  information  display- - This  CRT  presents  alpha- 
numeric information  (in  p>age  formats)  describing  training  in  progress,  the 
status  of  individual  trainee  stations  or  prevailing  environmental  conditions. 
A keyboard  and  light  pen  enable  the  instructor  to  access  the  computer  in 
order  to  modify  training  or  simulation  conditions  ( e . g . , malfunction  inser- 
tion/removal, modify  environmental  conditions,  select  and  reinstate  initial 
condition  parameters,  freeze  parameters  of  flight  individually,  etc.).  The 
functions  performed  with  this  c omputer  - generated  display  include  the 
following:  C . ~ 
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. Initialize  the  trainers 

select/modify  initial  conditions 
select  malfunctions 
modify  performance  alerts 

. Monitor  and  control  flight  parameters 

monitor  flight  parameter  status 
change/freeze/ restore  a flight  parameter 

* Monitor  and  display  radio  navigation  station 

# Recall  of  stored  plot  for  coaching/debriefing 

b.  Graphic  Plotter  Display- - This  is  a graphic  CRT  display 
(one  for  each  trainee  cockpit),  which  presents  a plot  of  ground  track,  time- 
based  plots  of  altitude  and  airspeed,  and  selected  status  information. 

c.  Closed  Circuit  Television  system  (CCTV)  — The  compo- 
nents of  this  cockpit  monitoring  system  are: 

. TV  camera  (one  in  each  cockpi*:} 

. Video  tape  recorder 

# CCTV  monitor  (one  in  each  cockpit  and  two 
at  the  instructor  console) 

d.  Record/ Playback  system- -This  equipment  records  student 
performance  for  the  purpose  of  playing  back  a copy  of  just  completed  per- 
formance (i.e.,  self-confrontation  technique)  . 

e.  Hard-copy  printout  system— This  information  is  used  for 
performance  assessment  in  the  critique.  A sample  information  content  in- 
cludes the  following: 

. Time  of  event  occurrence 

. Exercise  segment  in  which  the  condition  occurred 

. Difficulty  level  (adaptive)  of  exercise 

* Parameter  which  went  out-of-tolerance 

. Maximum  parameter  deviation  during  out-of- 
tolerance 
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• Time  (seconds)  difference  between  initial  and 
maximum  deviation  during  out- of- tolerance 

• Number  of  seconds  maximum  deviation  held 

• Number  of  seconds  required  for  aircraft  to 
return  to  in-tolerance  from  maximum  deviation 

f - X-Y  plotter- - This  is  used  in  making  permanent  records 
for  the  quality  control  of  performance  errors  for  each  training  session. 
Performance  error  parameter  totals  and  cumulative  times  will  be  recorded 
for: 


• Individual  error  parameters  regardless  of 
maneuver 

• Error  parameters  per  type  of  maneuver 

• Groups  of  trainees 

The  system  highlight  is  the  computer  programing  with  com- 
puter generated  CRT  displays  and  appropriate  controls.  Classes  of  per- 
formance status  and  errors  are  continuously  available.  Trainee  perfor- 
mance on  parameters  / maneuvers  is  compared  against  the  preprogramed 
ideal  performance  (for  given  parameters,  etc.),  plus  or  minus  the  toler- 
ances designated,  and  trainee  error  data  are  automatically  recorded  and 
available  for  display  either  as  accrued  or  on  demand.  Through  automated 
equipment  assists,  the  instructor  is  provided  opportunities  for  instructional 
control  and  for  developing  optimum  training  strategies  for  each  independent 
student  in  a multi- student  environment. 

3.5.3*  1.2  Simulator  for  Electronic  Warfare  Training  (SEWT).  A re- 

cent requirement  for  a Simulator  for  Electronic  Warfare  Training  (SEWT)* 
specifies  an  automated  monitoring,  evaluation  and  scoring  capability.  The 
highlights  of  this  capability  are  summarized  below  in  terms  of  a design 
philosophy  on  which  the  system  is  predicated. 

• Computer  programing  which  drives  the  simulator 
to  provide  a preprogramed  mission  scenario,  the 
monitoring  of  all  aspects  of  performance  and  relay 

^RFP  number  F33 657- 69 -R-0994 , Simulator  for  Electronic  Warfare,  U*S* 
Air  Force  Systems  Command,  Aeronautical  Systems  Division,  Wright  - 
Fatter  son  AFB , Ohio,  25  August  1969. 
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of  selected  knowledge  of  performance  information 
to  students,  and  hard-copy  records  of  performance 
both  for  in-mission  and  post-mission  critique  and 
for  school  records. 

• An  instructor  capability  to  monitor  and  evaluate  per- 
formance, and  provide  enroute  mission  instruction 
and  post-mission  critique  to  four  students  who  are 
performing  independently.  Computer- generated 

CRT  displays  provide  student  performance  information. 

• An  ability  to  present  preprogramed  standardized 
mission  scenarios  that  are  graded  in  difficulty  over 
the  course  of  instruction  and  which  can  be  modified 
during  each  training  session. 

• Automatic  evaluation  and  scoring  of  student  perfor- 
mance which  enables  the  instructor  to  develop  train- 
ing strategies  that  are  optimum  for  each  student. 

This  capability  permits  each  student  to  progress 

at  his  own  pace  in  achieving  the  training  objectives 
per  mission. 

• Flexibility  in  employing  the  simulator  for  training 
and  evaluation  purposes. 

Automated  evaluation  and  scoring  with  scoring  criteria  adjusted  to  the  stage 
of  training  of  each  student  provides  information  for  monitoring  and  error 
indications  which  are  displayed  to  the  instructor.  He  uses  these  directly  in 
evaluating  and  controlling  each  student’s  progress.  The  instructor  options 
in  developing  the  training  strategies  are  listed  below.  These  are  the 
changes  that  can  be  made  to  the  preprogramed  mission  either  by  event  or 
time  access  via  displays  and  controls  on  the  instructor  console.  Instan- 
taneous and  past-history  information  on  any  student's  performance  can  be 
called  up  from  storage. 

• Continue  preprogramed  mission,  no  action 
required. 

• Provide  verbal  feedback  of  performance  informa- 
tion to  selected  student  based  on  monitoring  in- 
formation obtained  from  CRT  displays.  This 
may  be  in  the  form  of  knowledge  of  results  of 
performance  or  cues  to  the  student  for  enhanc- 
ing  performance . 

361  ' 
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Freeze  the  problem  for  any  or  all  students  and 
provide  verbal  guidance,  or  freeze  the  problem 
until  all  students  have  completed  certain  require- 
ments; resume  mission. 

Demonstration  mode--Freeze  the  mission  either 
to  demonstrate  preprogramed  emitter  signatures 
to  all  students  or  to  demonstrate  specific  char- 
acteristics of  the  preprogramed  emitters  for  any 
or  all  students  when  performance  is  below  ex- 
pectations for  the  specific  mission  number;  resume 
mi  s s ion . 

Reinstruction  mode  — Freeze  the  mission  for  any 
or  all  students  when  performance  is  below  expectation 
for  the  specific  mission  number.  The  reinstruction 
requires  that  the  student  refly  a leg  or  phase  of  the 
mission.  The  mission  is  restarted  at  the  point  of 
the  refly  position.  This  mode  can  be  played  in  slow 
time  (l/4x,  l/2x)  and  in  real  time. 

Error  alert  mode  — When  a student  exceeds  the  error 
envelope  in  the  mission  fcr  a class  of  error,  an  auto- 
matic problem  freeze  occurs  at  the  time  the  next 
error  of  the  same  class  is  made  by  the  student.  The 
instructor  has  the  choice  of  manually  overriding  the 
computer  freeze  or  accepting  the  freeze.  His  in- 
structional options  are  those  indicated  above,  vis- 
a-vis  the  training  strategy  he  is  developing  for  the 
student  in  question. 

Insert  new  emitter  parameters — when  a student1  s 
performance  exceeds  the  mission  scenario  require- 
ments, the  instructor  has  the  option  of  inserting 
new  emitter  characteristics  in  addition  to  those  in 
the  preprogramed  scenario.  This  is  an  adaptive 
capability  by  which  problem  difficulty  can  be  in- 
creased to  keep  any  student  at  the  threshold  of  his 
ability  at  any  given  time. 

Insert  airborne  intercepts  (A/I*s)--two  classes  of  pre 
programed  A/Ts  are  available.  One  class  will  be  dis 
played  to  each  student  and  come  on  automatically 
unless  the  instructor  elects  to  override  this  event, 
causing  it  not  to  emerge.  The  other  class,  although 
programed,  will  not  come  on  unless  commanded  by 
the  instructor  when  the  acceptable  position  for 
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initiation  on  the  flight  track  occurs.  The  A/I  insertion 
can  be  controlled  to  achieve  the  standard  scenario  and 
also  can  be  adaptively  controlled  to  account  for  individual 
differences  in  skill  among  students. 

• Insert  in-flight  malfunctions  --  The  rationale  here  is  the 
same  a.s  for  the  airborne  intercept  insertion. 

The  heart  of  these  operations  is  centered  on  computer  pro- 
graming with  computer -generated  CRT  displays  and  appropriate  controls. 
An  alpha-numeric  CRT  presents  classes  of  student  errors,  as  accrued, 
continuously  throughout  the  mission.  A GRID  CRT  (i.e.  , CRT  with  light 
pen  control)  allows  a diversity  of  performance  information  to  be  displayed 
(both  alpha -numeric ally  and  pictorially)  via  standardized  CRT  page  formats 
Six  basic  display  and  control  modes  are  available. 

Monitoring  and  error  evaluation  mode 
Error  alert  mode 
Basic  monitoring  mode 
Demonstration  mode 
Airborne  intercept  control  mode 
Malfunction  insertion  mode 

In  essence,  the  instructor  is  provided  the  capability  for 
evaluating  performance  throughout  the  mission  to  obtain  information  on 
which  to  base  decisions  either  to  proceed  as  preprogramed  or  to  modify 
the  scenario  (in  the  events  presented  for  training)  or  to  modify  the  mission 
time  (via  problem  freezes).  The  structure  and  control  of  training  is  based 
on  determining  when  any  student  is  out  of  error  tolerance  with  the  perfor- 
mance standards  for  the  mission  being  flown,both  for  the  mission  preflight 
setup  procedures  and  when  operating  In  the  emitter  environment.  It  is  also 
based  on  determining  when  any  studentTs  performance  exceeds  the  perfor- 
mance standards  set  for  the  mission  being  flown. 

3. 5. 3.2  Design  Features  for  Implementing  Automatic  Scoring  in 
Training  Simulators.  This  subsection  considers  what  is  perhaps  the  most 
crucial  design  concern  of  the  immediate  future,  that  of  providing  an  auto- 
mated measurement  capability  in  training  devices.  The  development  of 
automatic  scoring  systems  must  resolve  a number  of  design  problems  to 
achieve  successful  implementation.  Unfortunately,  not  much  useful  data 
pertinent  to  these  issues  are  available.  Guideline  data  for  design  speci- 
fication are:  1)  notably  absent  because  the  published  research  has  not 

dealt  directly  with  the  use  and  implementation  of  automated  scoring  for 
purposes  of  training,  or  2)  difficult  to  provide  because  of  a number  of 
system- specific  requirements  that  are  pertinent  to  any  given  training 
simulator.  In  a real  sense,  these  issues  generate  the  need  for  consid- 
erable research  to  resolve  a number  of  difficult  decision  points  for  design. 
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Thus,  our  approach  is  to  identify  the  classes  of  problems  that 
must  be  resolved  and  what  actions  are  required.  The  design  issues  which 
follow  are  among  those  of  greatest  significance  to  the  design  of  automatic 
monitoring,  evaluation  and  scoring. 

3. 5. 3.2.  1 Data  Requirements  for  Derived  Measures.  An  initial  effort 

is  needed  to  determine  the  raw  data  requirements  from  which  derived  scores 
will  be  obtained.  The  basic  approach  is  to  develop  quantitative  expressions 
based  on  training  requirements  information.  These  requirements  are 
system- specific , correlated  with  the  class  of  training  device  under  con- 
sideration and  with  the  training  objectives.  The  parameters  to  be  sampled 
describe  the  interrelations  among  own  vehicle,  the  target  and  the  media. 

The  parameters  recorded  relate  to  classes  of  data  specific  to  types  of  train- 
ing devices.  For  example,  vehicle  management  parameters  are  of  most 
importance  to  instrument  flight  trainers  whereas  tactical  employment  and 
load  (e.g.  , vehicle  units  in  an  engagement)  parameters  are  of  significance 
to  tactical  team  training  devices.  The  relevant  parameters  reside  in  the 
following  classes  of  data. 

a.  Vehicle  management- - This  class  of  recorded  data  refers 
to  vehicle  positioning  and  power  plant  management  parameters.  For  ex- 
ample, for  device  2B24  (helicopter  instrument  flight  trainer),  the  perfor- 
mance parameters  to  be  recorded  during  automatic  mode  operation  include: 

• airspeed 

• altitude 

« flight  path 

• rate  of  climb 
« rate  of  turn 

. pitch  attitude 
. bank  attitude 
. rotor  RPM 

• gas  produced  RPM 
. EGT 

b.  Relative  geometry  considerations -- This  class  of  data  is 
concerned  with  own- vehicle  position  and  action  direction  or  with  an  em- 
placed unit  (e.g.  , carrier  aircraft  controller)  in  relation  to  other  units/ 
vehicles  in  an  area  of  engagement  or  involvement.  For  example,  in  surface 
or  subsurface  vehicles  the  important  parameters  include  the  variations  in 
range  and  bearing  data  (course,  speed,  depth  maintenance  or  change),  and 
rate  of  change  of  these  relationships  (closing,  opening  rates  , etc.).  For  a 
fixed  or  emplaced  unit  (GCA  or  carrier  control  of  aircraft)  the  parameters 
of  concern  include  the  specific  aircraft  in  relation  to  the  recovery  site  and 
the  positions  and  movements  among  aircraft  in  a defined  pattern. 
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c.  Tactical  employment  of  vehicle -- The s e data  refer  to  the 
relevant  parameters  in  the  basic  detection-to-engagement  paradigm,  in- 
volving own  vehicle,  friendly  unit(s)  and  target  vehicle(s).  For  example, 
in  surface  ship  ASW  training,  the  relevant  parameters  inhere  in  sonar 
detection  and  classification,  in  attack  director  operations  involved  in  de- 
veloping the  fire  control  solution,  and  in  certain  Combat  Information  Center 
operations  (e.g.  , CIC  plots)  in  support  of  operating  the  ship  as  a fighting 
unit . 

d.  L,oad--These  data  relate  to  the  amount  of  Mcluttern  (i.e.  , 
number  of  things)  involved  in  a training  situation,  and  include,  the  number 
of  different  units  in  an  engagement  (support  units,  friendly  units,  targets), 
command  relationships  (HUK  forces,  SAUs  , etc.),  and  communications 
requirements  . 


e.  Weapons  employment — These  data  refer  to  the  parame- 
ters recorded  on  weapon  selection  and  preparation  and  on  relevant  weapons 
actions  (e.g.  , water  splash  point  for  surface  to  subsurface  missile,  torpedo 
launch  and  course;  countermeasures). 

3. 5. 3. 2. 2 Selection  of  Measures  . The  measures  selected  are  based  on 

the  specification  of  the  knowleclge  or  information  wanted  about  performance. 
Initially,  measurement  is  concerned  with  determining  that  the  critical  task 
events  have  occurred  correctly.  In  inadequate,  then  diagnostic  measures 
are  needed  to  indicate  the  nature  of  faulty  performance.  Usually,  well- 
stated  training  requirements  suggest  quite  easily  the  corresponding  meas- 
ures and  specific  performance  criteria.  Measurement  must  determine  if 
these  criteria  have  been  met  within  the  tolerances  specified.  The  param- 
eters selected  are  scored  and  compared  to  some  standard  to  derive  the  type 
of  measure  desired.  One  of  the  problems  in  programing  the  computational 
system  is  determining  what  aspects  of  performance  are  to  be  included  in  the 
measure . ;■ 

Derived  measures  are  specific  to  the  types  of  training  sys- 
tems, i.e/,'  those  having  similar  training  purposes.  Obviously,  a variety 
of  measures  are  available  and  no  useful  ways  are  suggested  economically 
for  capturing  and  organizing  the  types  and  arrays  of  measurement  possi- 
bilities. Thus,  our  approach  to  the  discussion  of  measure  selection  for 
machine  scoring  will  center  on  classes  of  measures  pertinent  to  types  of 
training  devices  in  the  NTDC  inventory.  In  essence,  these  classes  of 
measures  can  be  organized  as  follows: 

m Manual  control  of  vehicles --Measures  of  vehicle  posi- 
tioning and  tracking,  primarily  flight  vehicles  with 
some  concern  for  surface  and  undersurface  vehicle 
control  (e.  g.  , maneuvering,  docking,  etc.). 
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. Procedures  following- -Measures  pertinent  to  any 
training  device  class  in  terms  of  time  / event  se- 
quences (errors  of  sequence,  omission  and  commission 
of  activities,  and  inappropriate  timing  of  activities). 

# Decision  making  - -Measures  pertinent  to  actions 
involved  in  tactical  operations  (e.g.  , search  and 
attack  options  in  an  ASW  course  of  action),  and 
measures  pertinent  to  individual  performance 
relative  to  tactical  employment  of  a vehicle. 

« Team  performance  - -Measures  pertinent  to  the 
coordinated  performance  of  groups  in  defined 
mission  contexts. 

a.  Manual  Control  of  Vehicles  - -Several  kinds  of  single  axis 
error  amplitude  scores  are  widely  employed  in  describing  manual  control 
(tracking)  performance.  Those  used  most  prominently  are  proportion  of 
time-on-target,  root  mean  square  error  (RMS),  average  error,  and  aver- 
age absolute  error.  Table  27  indicates  the  principal  error  amplitude  scores 
and  corresponding  equations  for  discrete  (non- tracking)  measures. 

Kelley  (1969a),  states  that  assuming  a Gaus  sian  distribu- 
tion, RMS  error  is  the  most  reliable  estimate  of  dispersion  since  this  score 
has  the  least  relative  sampling  error  of  the  measures  of  dispersion.  Aver- 
age absolute  error  is  also  a reliable  measure  of  dispersion.  Proportion  of 
time  on  target  scores  are  less  reliable  than  the  former  two.  Reliability  is 
influenced  by  the  size  of  Ptot  U-e*  * target  zone).  Bahrick,  Fitts  and 
Briggs  (1957)  indicate  that  summed  time -on- tar  get  yields  a measure  which 
varies  nonlinear ly  with  the  size  of  the  target  area  (i.e.  , the  shape  of  the 
learning  curve  changes  nonlinearly  as  target  size  changes).  Thus,  too  low 
or  too  high  a time -on- tar  get  score  range  in  undesirable.  They  suggest 
that  the  optimum  target  zone  includes  about  68%  of  the  tracking  distribution. 

There  are  various  techniques  for  combining  error  ampli- 
tude scores.  The  most  important  of  the  multi-axis  error  amplitude  scores 
is  shown  in  Table  28.  Kelley  ( 1969a)  provides  a detailed  discussion  of  the 
characteristics  of  these  measures  and  the  conditions  for  their  use. 
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The  frequency  characteristics  of  tracking  performance 
may  be  used  to  yield  information  on  effort  expenditure  in  performance. 

Two  performances  may  be  the  same  in  terms  of  some  amplitude  score  yet 
may  be  quite  different  in  the  amount  of  effort  in  achievement.  Frequency 
measures  also  provide  indications  of  how  the  trainee  is  solving  the  control 
problem.  Simple  frequency  measurements  include  zero  crossing  error, 
zero  crossings  of  the  derivatives  of  error.,  ^nd  the  instances  when  a defined 
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TABLE  27.  Fundamental  Equations  for  Continuous  Tracking 
Scores,,  with  the  Analogous  Equations  for 
Discrete  Measurements. 


Tracking  Score 

Discrete  Score 

Mean 

(Xd-X)dt 

( usually  assumed  to  be  zero  in  tracking) 
rms  Error 

Mean 

x=±yx. 

Standard  Deviation 

Er».  = \/ ~ XY  dt 

A verage  Absolute  Error 
\E(i)\  = jja  \X*-X\ dt 

Proportion  of  Time  on  Target 

P _ 

" tot  rp 

Average  Deviation 
AD  = ±y\X-Xi\ 

Proportion  of  Scores  within  Tolerance 

P — «*• 
r»wt  — ■ 

n 

Legend 

Continuous 

Discrete 

X ===  X(t)  = a continuous  tracking  sample 

Xi,  X2j ...  ,Xn=  a sample  of  discrete  scores 

T = length  of  tracking  sample 

n = number  of  scores  in  sample 

Xd  ss  Ad(r)  = desired  value  of  AT;  when  Xd 

varies  with  time,  Xd(t)  is  the 

forcing  function  that  is  tracked 

E = Xd  — X 

| X — Xi\  = deviation  (absolute)  from 

| | = | Xd  — X | = absolute  error 

the  mean 

Tot  = length  of  time  on  target  during 

nw  t = number  of  scores  within 

trial  of  length  T 

tolerance 

(from  Kelley,  196  9a) 
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TABLE  28.  Multi- Axis  Error  Amplitude  Scores 


Weighted  Time-Averaged  Error  for  n Independent  Axes 
— 1 fT 

Em  J,  J (#  1 -Si  -j-  £22  S'2  -{-  . . . -j-  an  sn)dt 

n 

= ^atS, 

4=1 

Independent  axis  errors  may  be  summed  and  then 
time-averaged  or  time-averaged  and  then  summed. 


Instantaneous  Vector  Error  in  n Axes  (Scaled) 

Ev=  Va?  ES  + a?  E?  -f  . . . + a7  Et? 


Multi-Axis  Time-Averaged  Vector  Error  (Scaled) 

_ i rr 

E*  = j;J0 

= j;J  * ( v'fli3  Ei-  + a?  E?  + . . . + E7)dt 


Multi-Axis  rms  Vector  Error 


Evrtaa  J^/  rj,  J"  (fil2  El2  -j-  az~  E 2 -j-  . . . -}-  dn2  En 


)dt 


Multi-Axis  Time  on  Target 


Xmot 


Multi-Axis  Vector  Error  Time  on  Target 


Employs  equation  for  single-axis  time  on  target, 
with  the  target  band  defined  in  terms  of  Ev,  the 
instantaneous  vector  erron  To  avoid  the  con- 

tinuous square  root  operation,  the  square  of  the 
vector  error  may  be  employed  and  compared  to  the 
square  of  the  desired  target  zone. 


Time-Averaged  “Largest  Error”  Multi-Axis  Score 


=U' 


Etn  d t 


Legend 


£i,  ^2, . . . ,sn  = instantaneous  individual  axis  errors  (|£T|,  or  E °) 

Si,  s2, . . .,sn~  time-averaged  error  for  each  axis, 
aii  0.2., . . . , an  ~ weighting  or  scaling  coefficients  for  the  scores  in  each  axis 
T = length  of  the  tracking  trial 
Ex  ~ instantaneous  error  in  the  ith  axis 
Pm  tot  = proportion  of  time  on  target  in  all  axes  simultaneously 
Tmot  = length  of  time  when  no  axis  was  out  of  tolerance  during  trial  o: 
length  T 

Em=  value  of  the  largest  absolute  instantaneous  error  of  any  axis 


(from  Kelley,  1969a.) 
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criterion  of  error  is  exceeded.  (See  Fitts,  Bahrick  and  Bennett,  1956; 
Obermayer,  Swartz  and  Muckier,  1962.) 

There  are  also  measurement  techniques  employed  by 
control  engineers  for  measuring  tracking  performance.  These  include 
the  measurement  of  input-output  relations  (describing  functions),  cross- 
over models  relating  frequency  and  amplitude  of  response,  and  parameter 
adjustment  for  model  matching  (match  between  output  of  the  human  and  that 
of  the  model).  Reviews  of  these  techniques  and  other  approaches  can  be 
found  in  Kelley  ( 1969a)  , and  McRuer  and  Jex  ( 1 967)  . 

b.  Procedures  following -- This  general  class  of  measure- 
ment is  pertinent  to  all  types  of  training  devices  and  basically  involves  the 
adequacy  of  time  and  event  sequences  in  task  or  mission  performance.  The 
scoring  concerns  errors  in  sequencing,  omission/ commission  of  activities, 
and  inappropriate  timing  of  activities.  The  common  usage  of  time  and 
event  scoring  and  recording  (e.g,  , time  and  event  hard-copy  printouts) 
precludes  the  need  for  much  discussion.  One  example  is  provided  concern- 
ing Electronic  Warfare  Training  in  the  emitter  environment  phase  of  a de- 
fensive (airborne)  mission.  An  example  of  the  measures  that  may  be  com- 
puter scored  is  shown  below. 

Switch  setup  in  signal  analysis 

Switch  position  in  detection  (signal  to  be  countered) 

Time  to  detect 
Failure  to  detect 

Cue  buildup  (onset  of  signal  to  action) 

Switch  position  in  identification  (wrong  emitter) 

Time  to  identify  type  of  signal 
Mode  of  operation  in  switch  settings 

receiver 

transmitter 

expendables 

Malfunction  correction  sequences 
Total  number  of  signals  countered 

Threat  assessment 

Switch  positions 
Time  of  handling  threat 
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Setup  for  jamming 


Switch  positions  on  jamming  systems 
Time  to  set  up 

Initiation  of  jamming 


Time  at  which  jamming  is  initiated 
Time  during  which  jamming  is  effective 

Use  of  expendables 

Switch  positions  for  selecting  expendable  dispensing 
programs 
Time  to  initiate 

Time  at  which  expendable  program  is  terminated 
Failure  to  release  expendables 
Maintenance  of  minimum  stores  level 


Threat  priority 

Signals  attended  to  in  time 
Switch  positions 

Time  of  handling  new,  higher  priority  threat  (cue  buildup) 

Environment  Coverage 

Signals  or  bands  covered 
Jammer  refinement  switch  settings 

initial 
subs  equent 

Number  of  jammers  on  individual  signals  or  bands 

Evasive  ma  ne  uvers 

Maneuver  selection 
Time  of  initiation 

c.  Measures  describing  tactical  decisions --Of  concern  here, 
are  measures  appropriate  to  the  tactical  employment  of  vehicles,  particu- 
larly as  applicable  to  surface  and  subsurface  tactical  team  trainers . At 
' present,  no  provision  is  made  in  these  training  devices  for  an  automatic 
measurement  capability  (see  paragraph  3.5.1). 
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Tactical  decision-making  presents  difficult  measurement 
problems,  primarily  due  to  the  nature  of  decision-making  activity.  While 
the  antecedent  operations  are  complex,  the  overt  acts  are  relatively  simple. 
It  is  also  unlikely  that  a given  tactical  decision  can  be  judged  right  or  wrong 
in  the  absolute  sense  (except  in  the  event  of  obvious  errors  based  on  the 
relative  geometry  of  a situation).  Consider,  for  example,  the  case  of  own- 
ship  holding  contact  on  a hostile  target  where  own-ship's  weapons  have  a 
greater  probability  of  kill  as  range  is  closed.  Obviously  the  closer  own- 
ship  gets  to  the  target,  the  more  likely  is  a successful  attack,  but  the  more 
likely  is  own-ship  to  be  attacked  by  the  target  or  the  more  likely  is  the 
target  to  carry  out  its  primary  mission.  In  such  a situation  there  is  no 
obvious  single  ''right  time"  to  attack.  (Thus,  the  outcomes  of  the  decisions 
must  be  explored  to  identify  the  factors  involved  in  reaching  the  decision.) 
Similarly,  in  search  and  attack  operations,  there  may  be  several  equally 
good  solutions  based  on  the  combinations  of  search  and  attack  procedures 
employed.  The  point  to  consider,  however,  is  that  there  are  but  a limited 
number  of  "good"  solutions  and  also  obviously  incorrect  solutions  for  a 
given  situation. 

Several  classes  of  quantitative  part-task  measures  can  be 
employed  to  achieve  indications  of  the  adequacy  of  tactical  operations.  To 
be  successful,  they  must  be  based  on  precise  mission  models  of  engage- 
ments which  identify  the  lawful  alternatives  for  success.  These  classes  of 
measures  which  are  mission- specific  in  their  detail  are  as  follows: 


0 
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. Positioning  of  vehicles --The se  involve  time  history 
plots  of  relative  geometry  based  on  course,  speed, 
depth/altitude  changes  (e.g.,  maintenance  of 
positions,  closing/ opening  rates  involving  bearing 
and  range  changes). 

Time  and  event  happenings -- These  refer  to  mission 
events  upon  which  decisions  are  based,  e.g.  , sonar 
detections  and  clas sifications  > radar  detections, 
achievement  of  fire  control  solutions,  CIC  evalua- 
tions, etc  . 

. Selection  of  weapons  or  of  a course  of  action-- These 
involve  the  selection  and  employment  of  resources 
(launch  time  of  specific  weapons  , release  of  counter- 
measures, etc.),  or  the  selection  of  available  (de- 
fined) search  options  (e.g.  , ASW  "Oak  Tree"  search 
pattern)  and  attack  options  (e.g.  , "Geo  Sector" 
ASROC  launch)  . 
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• Coordination  with  other  units --These  refer  basically  to 
inter -ship  communications  involving  type  of  and  time  of 
messages  disseminated. 

• Appropriate  response  sequences -- The  in-tolerance  re- 
sponse sequences  must  be  correlated  with  the  mission 
model  since  there  may  be  a finite  number  of  equally 
good  sequences  to  achieving  terminal  solutions. 

d.  Team  Performance — The  measurement  of  team  perfor- 
mance is  a difficult  undertaking  for  a computer  scoring  system.  A basic 
problem  is  the  manner  in  which  team  training  is  conceptualized.  When 
team  interaction  is  viewed  as  a sequence  of  properly  planned  and  executed 
individual  actions,  that  is,  a job  position  orientation  with  explicit  tasks 
assigned  to  each  member  and  with  sequences  of  performance  required  to 
carry  out  defined  mission  tasks  (e.g.  , two  or  more  man  air  crews),  then 
measures  of  system  output  may  be  sufficient.  When  team  coordination, 
however,  refers  to  the  extent  to  which  members  interactively  perform  in 
situations  where  there  are  no  predetermined  standards  of  performance 
(i.e.  , team  operations  characterized  by  emergent  situations  (Boguslaw  and 
Porter,  1963),  then  procedures  tend  to  be  developed  by  the  team  rather 
than  imposed  on  the  team.  Teams  may  be  equivalent  in  overall  performance 
yet  differ  considerably  in  procedures  and  in  self- or ganization . Several 
equally  good  solutions  to  problems  usually  exist  and  inter-team  differences 
in  procedures  become  evident  in  coping  with  the  environment.  When  the 
development  of  coordinative  skills  is  stressed  based  on  adequate  individual 
skills  in  each  trainee’s  position,  measurement  is  difficult  and  unreliable. 

A summary  of  team  training  assessment  attempts  can  be  found  in  Smode, 
Hall  and  Meyer  ( 1966)* 


One  attempt  at  describing  this  performance  has  been 
through  examining  team  communications  (the  equipment  requirements  for 
this  involve  only  an  audio  recording  and  controlled  playback  capability). 

A study  by  Krumm  and  Farina  (1962)  developed  communication  measures 
as  one  means  of  defining  crew  coordination  in  the  B-52  integrated  simula- 
tor facility  (an  MB-41  flight  simulator  electronically  linked  to  an  APQ-T2A 
Ultrasonic  Bomb-Nav  trainer  with  a crew  of  two  pilots  and  two  navigators). 
Measures  were  taken  of  volume  (production  of  message  units)  and  pattern 
(kinds  of  message  units  such  as  voluntary  inputs,  acknowledgements,  etc..). 
The  communication  scores,  however,  were  found  not  to  be  unequivocally 
acceptable  as  a criterion  measure.  Siskel,  Lane,  Powe  and  Flexman  ( 19 65) 
investigated  communication  processes  in  actual  B-52  and  KC- 135  flights. 
The  mission  segments  chosen  for  analysis  were  the  takeoff  and  the  bomb 
run  for  the  B-52,  and  takeoff  and  aerial  refueling  for  the  KC-135.  The 
hypothesis  that  more  experienced  crews  would  show  lower  communications 
rates  (transmissions  and  messages)  thau  less  experienced  crews  was  not 
confirmed. 
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Studies  by  Federman  and  Siegel  (1965)  and  Siegel  and 
Federman  (1968)  examined  the  communications  of  anti-submarine  warfare 
(ASW)  helicopter  crews  (pilot,  nav-copilot  and  sonar  operator)  while  per- 
forming tactical  flight  problems.  In  the  first  study,  two  tactical  flight  prob- 
lems were  developed  for  use  in  device  15R10  (ASW  helicopter  trainer).  Data 
were  collected  on  twelve  helicopter  teams  (a  team  consists  of  two  crews). 

Fourteen  unitary  communications  predictors  of  performance 
proficiency  were  isolated  and  found  to  be  directly  related  to  miss  distance 
(i.e.  , distance  of  weapon  impact  point  from  target).  These  communications 
variables,  presented  as  ratios,  were: 

Activity  messages/N^ 

Evaluative  messages/N 

Confusion  risk  willingne  s s / Reluctanc  e to  confusion 
risk  willingness 

Directing  messages  / Requests  for  directions 
Nonrequested  messages/N 
Phenomenological  messages/N 
Invitational  messages/N 

Phenomenological  and  invitational  me  s sages / Objective 
messages 

Progressive  me s s ages / Regr  e s sive  messages 
Requests  for  infor mation/N 
Provides  information/ N 
Requests  for  opinion/N 
Provides  opinion/ N 
Voluntary  opinion/N 

A factor  analysis  of  these  14  unitary  communications  predictors  yielded 
four  factors  which  were  labeled:  probabilistic  structures;  evaluative  inter- 

change; hypothesis  formulation;  and  leadership  control.  The  second  study 
replicated  the  first  study  but  used  device  14H4,  ASW  Helicopter  simulator. 
The  findings  of  the  first  study  were  verified  and  four  additional  unitary 
communications  predictors  were  identified.  These  were: 


, 
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evaluative/activity 
concordant/ discordant 
corroborated/ N 
extrapolation/ N 

Three  of  the  four  factors  identified  in  the  first  study  were  found  to  be  con- 
gruent: the  fourth  factor  , hypotheses  formulation,  was  not  confirmed  by  the 

^N  is  equal  to  the  total  number  of  goal-oriented  communications. 
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second  study.  Based  on  this  verification  of  the  generality  of  the  factors, 
a course  of  training  for  A5W  helicopter  pilot  in  the  use  of  the  emergent 
communications  factors  was  developed,  administered  and  evaluated.  The 
logic  underlying  the  research  is  that  measurable  communications  entities 
exist  which  may  be  employed  in  improving  ASW  proficiency. 


As  a group,  the  various  researches  on  communications 
measures  as  indicants  of  team  performance  have  not  yielded  much  informa- 
tion for  measurement  purposes  and  the  value  of  this  approach  for  automated 
measurement  is  unclear.  Perhaps  the  most  useful  statement  to  be  made  is 
that  team  coordi native  measures  of  performance,  other  than  measures  of 
system  output,  do  not  lend  themselves  to  automatic  scoring,  and  are  best 
accounted  for  in  the  post-mission  critique  session  via  instructor  judgments. 

3. 5. 3.2. 3 Data  Sampling  Rates.  An  important  design  consideration  is 

performance  sampling  rate.  The  issues  concern:  how  often  the  measure- 

ment data  are  sampled  in  the  training  exercise;  the  length  of  time  each 
sample  is  examined;  and  the  places  in  the  mission  scenario  wherein  the 
sampling  is  conducted.  These  decisions  are  correlated  with  computer 
"space  availability1'  for  the  purpose  of  scoring  performance.  Logically, 
the  sampling  time  for  defined  tasks  (e.g.  , a 90°  level  turn  at  a rate  of  3°/ 
sec)  is  set  to  include  that  complete  performance;  for  more  or  less  continu- 
ous controlling  (e.g.  , maintaining  altitude  or  heading  in  turbulence)  , the 
sampling  time  must  be  based  on  other  logical  considerations,  derived 
usually  from  experience.  For  example,  the  SFTS  (device  2B24)  is  pro- 
gramed to  measure  performances  in  ten-second  interval  samples  . The 
question  of  the  reliability  of  the  sampling  intervals  must  be  considered 
carefully  in  terms  of  minimum  man-system  response  times  and  in  terms 
of  specific  task  characteristics  (e.g.  , aircraft  roll  oscillations  of  three 
seconds  per  cycle  should  be  sampled  for  several  complete  cycles  per 
interval)  . 


3. 5. 3.2.4  Error  Tolerance  Criteria.  Means  are  required  for  inserting 

error  tolerance  criteria  into  the  computer  (pre-mission)  and  also  to  modify 
error  criteria  during  a mission,  as  required.  For  example,  with  prepro- 
gramed scenarios,  error  tolerance  criteria  may  require  modification 
manually  on-line  as  a function  of  student  performance  per  stage  of  training. 
In  the  case  where  a student  exceeds  the  preset  error  tolerance  and  a com- 
puter freeze  occurs  (i.e.,  an  error  alert  mode)  , the  means  must  be  pro- 
vided via  keyboard  address  to  the  computer,  etc.)  to  override  the  previous 
error  criteria  and  modify  (as  required)  by  inserting  new  tolerances  to 
enable  the  continuation  of  the  exercise  according  to  the  training  strategy 
elected  by  the  instructor  . 5 : 


The  question  of  suitable  criteria  to  use  for  error  determina- 
tion in  the  computer  is  an  issue  of  concern  and  guidelines  for  design  are 
indeterminate.  Most  often,  these  criteria  are  based  on  what  subject  matter 


- r"  ■ 

- 362  - 


V ?'j  - 


1.  • 

- 

NAVTRADEVCEN  69-C-0298-1 


! 

I 


i 

s 


! 


experts  consider  good  performance  (i,  e.  , preconceived  notions  of  ideal 
performance).  However,  criteria  should  also  be  based  on  what  good  job 
incumbents  actually  do  during  task  operations.  Criteria  established  on 
this  basis  requires  the  collection  of  normative  data  from  samples  of  actual 
job  performance.  Fortunately,  computer  programing  techniques  enable 
error  criteria  modification  on-line  as  experience  with  the  training  system 
accumulates.  Even  if  specific  values  for  parameters  are  unavailable, 
memory  space  can  be  reserved  and  used  as  data  are  collected. 

3. 5. 3.2. 5 Organizing  the  Measurement  Data . The  computer  is  utilized 

for  scoring,  evaluating  and  recording  performance.  Time  histories  of  per- 
formance are  easily  obtained  as  is  information  on  event  or  status  situations 
due  to  their  binary  nature.  Frequency  data  are  likewise  easily  assembled 
and  manual  control  scores  (e.g.  , integrated  error  rates)  can  be  quickly 
computed.  Larger  segments  of  performance  and  terminal  (system)  outputs 
are  also  achievable  directly.  The  computer  is  also  able  to  combine,  inte- 
grate and  weight  scores,  and  compare  the  results  against  defined  error 
tolerance  criteria.  Computer  - gene  rated  displays  and.  hard-copy  perfor- 
mance output  records  are  available  to  present  feedback  of  performance 
information  to  the  trainee  and  performance  and  error  information  for  in- 
structor monitoring  through  output  devices.  The  instructor  is  also  able  to 
perform  as  a link  to  the  computer  for  trainee  performances  not  easily 
processed  automatically  and  he  is  able  to  access  the  computer  directly  for 
performance  error  and  status  information  via  input  devices  (e.g.  , type- 
writer, light  pen).  In  essence,  automatic,  monitoring,  evaluation  and  scor- 
ing can  be  accomplished  with  facility  . Thus  , the  design  decisions  to  be 
made  concern  the  following: 

. Combining,  integrating  the  performance  dimensions/ 
parameters  scored. 

# Weighting  of  scores  and  priority  ranking  of  scored 
performance  for  error  and  situational  display  at 
the  instructor  station  . 

# Providing  feedback  of  performance  information  to 
the  trainee  (error  presented  immediately  as  accrued, 
on  primary  or  secondary  displays  in  the  trainee  station). 

# Display  of  performance  information  (error  and 

situational)  via  computer-generated  CRTs  at,  the 
instructor  station.  Care  must  be  exercised  in 
specifying  the  performance  information  to  be 
displayed  in  terms  of  successive  page  formatting 
(alpha-numeric  and  situational  display  of  error  and 
status)  for  the  various  instructional  modes  (e.g.  , 
monitoring  and  control,  evaluation,  demonstration, 
reinstruction).  yy.-/ 
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. Formatting  requirements  for  retrievable  (hard-copy) 
records  of  all  error  scoring  and  the  time  and  event 
happenings  throughout  the  exercise.  The  permanent 
record  is  of  two  types:  student  critique,  and  school 

record-keeping.  Since  the  computer  is  capable  of 
recording  all  time  and  event  happenings  that  occur 
in  the  mission,  all  factors  that  relate  to  human 
performance  can  be  provided  via  hard-copy  printout. 
This  printout  can  be  to  the  level  of  any  switch  which 
is  changed  by  the  student  in  the  exact  sequence  of 
manipulation  or  only  to  the  level  of  significant  action 
events  . Any  level  of  call  out  can  be  specified. 

Time  and  Event  Printouts  — present  the  total 
performance  indicants,  and  are  used  in  school 
r e co r d- keeping  functions.  An  additional  use  is 
in  evaluation  (checkride)  missions  where  detailed 
indications  are  desired  on  the  progress  and  out- 
come of  the  mission. 

Total  Error  Printouts  - -these  are  complete  error 
printouts  across  mission  time  and  are  used  for 
critique  of  training.  Error  indications  are  pre- 
sented by  classes  and  trends  of  errors  are 
identified. 

- Summaries  of  Error  Scoring  — these  are  also 
used  for  critique  purposes,  in  the  instructional 
sense,  since  they  represent  short  summaries 
of  errors  made  per  error  class,  scores  achieved 
per  mission  segment,  total  score  for  the  mission, 
etc.  The  value  of  this  type  of  single-page  summary 
per  mission  is  that  a file  on  each  student  can  be 
kept  simply  which  can  be  used  in  diagnosis  of 
performance  error  as  well  as  for  trend  depiction. 


3.5.3.Z.6  Summary.  The  achievement  of  automated  performance 

measurement  systems  in  simulators  has  been  hampered  by  major  obstacle 
A recurring  problem  has  been  criterion  determination,  that  is,  the  need, 
for  quantitative  baseline  values  with  which  performance  data  may  be  com- 
pared for  purposes  of  evaluation.  Another  major  problem  has  been  the 
determination  of  what  measures  best  describe  performance  and  what  vari- 
ables relate  significantly  to  effective  performance'.’-  Buckhout  and 
Cotter  man  (1963)  enumerated  some  key  (and  still  valid)  requirements  for 
the  development  of  an  automated  scoring  capability.  Beginning  with  an 
understanding  of  the  purpose  and  the  use  of  the  scores , defining,  and 
c la. s s ify ing  the  behaviors  to  be  measured  is  an  initial  undertaking.  Means 
for  quantifying  these  behavioral  elements  must  be  established.  Thus,  the 
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job  must  first  be  partitioned  into  manageable  units  of  behavior  (tasks,  task 
elements,  etc.)  which  are  observable  and  (in  most  instances)  quantifiable. 
Criteria  or  standards  of  performance  must  then  be  developed  since  quanti- 
fying  behavior  involves  examination  of  mission  requirements  and  selection 
of  those  parameters  which  are  useful  in  evaluation  and  which  can  be  effi- 
ciently obtained.  Following  this,  the  scores  must  be  tested  to  see  how  well 
they  predict  performance,  i.e.  , was  the  correct  selection  of  scores  made? 
Finally,  how  well  the  scores  stand  up  under  repeated  use  must  be  deter- 
mined. This  systematic  development  is  a big  order  , since  methodologies 
are  lacking  to  accomplish  each  of  the  requirements  completely.  The  prob- 
lems of  automatically  scoring  the  critical,  meaningful  aspects  of  complex 
tasks  have  not  yet  been  thoroughly  analyzed.  The  research  needs  include: 
selecting  or  developing  recording  procedures,  selecting  measures,  devel- 
°ping  testing  programs,  determining  the  applicability  of  measurement 
operations  to  task  structure,  and  predicting  mission  performance.  It  is 
not  simply  a problem  of  identifying  and  using  available  scoring  equipments. 

3.5.4  Automation  of  Instructor  Functions.  An  emerging  theme  in  th©  de- 
sign for  the  management  of  training  in  simulators  is  the  automatic- 1?.  pt  in* 
structor  functions.  This  is  a design  goal  that  is  just  beginning  to 
ined  in  terms  of  implementation . It  merits  a notation  here  since  is  concept 
and  approach  to  training  system  design  is  predicated  on  an  automatic  moni- 
toring, evaluation  and  scoring  capability.  Two  research  programs, 
currently  underway  , both  utilizing  the  flight  simulation  context  are  provided 
here  as  examples  of  this  innovation  of  the  future. 

The  feasibility  of  developing  design  guides  for  automating  various 
instructor  functions  has  been  reported  by  Leonard,  Doe  and  Hofer  (1969). 
This  study  investigated  the  problems  involved  in  building  an  automated 
weapon  system  trainer  with  the  objective  of  determining  what  training 
functions  could  be  automated  within  limits  set  by  the  state-of-the-art  in 
computer  technology  and  in  training  theory.  The  F-4  aircraft  character- 
istics and  procedures  were  utilized  as  the  basis  for  this  study . Five  speci- 
fic flight  segments  were  analyzed  in  terms  of  training  needs  and  a training 
approach  was  developed  for  each  of  these  flight  s e gments  . The  flight  seg- 
ments were:  instrument  flight  maneuvers;  ground  controlled  approach; 

offset  bombing;  navigation;  and  takeoff,  climb  and  emergency  procedures  . 
For  each  of  these  segments,  techniques  were  described  for  structuring 
training  sequences,  based  on  a WST  program  language  (WPL)  . Design 
guides  were  developed  for  automatic:  trainee  performance  monitoring; 

trainee  performance  evaluation;  task  difficulty  modification  (adaptive); 
and  exercise  preparation*.  In  essence,  the  steps  in  the  development  of 
the  program  language  is  shown  in  Figure  43.  The  authors  believe  that, 
T,although  nearly  every  aspect  of  training  can  now  feasibly  be  automated, 
the  major  job  of  effectively  doing  so  is  very  complex,  even  though 
straightforward.  This  complexity,  coupled  with  an  unavoidable  uncertainty 
about  human  behavior,  implies  that  training  program  construction  will  not 
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come  about  without  many  false  starts.  Thus,  the  first  cut  training  program 
can  be  expected  to  have  mistaken  assumptions  about  trainee  learning  capa- 
bilities and  problems.  A workable,  completely  automatic  training  program 
must  be  flexible  and  contain  techniques  for  rapidly  altering  that  program 
once  built.  M 

,lThe  WST  Programing  Language  (WPL)  discussed  in  this  report 
would  provide  the  training  technologist  with  a tool  for  specifying  and  alter- 
ing training  approaches  in  a rapid,  flexible  manner.  WPL  could  be  an 
immense  aid  to  creating  the  kind  of  troubleshooting  and  performance-  . 
criteria-development  capabilities  that  are  required  in  a truly  automatic 
system.  M (p.  17  3.) 

A study  by  Vreuls  and  Obermayer  (1970)  examined  the  feasibility 
of  developing  methods  and  measures  to  satisfy  requirements  for  an  auto- 
mated weapon  system  trainer.  The  goal  was  to  establish  the  state-of-the- 
art  in  measurement  upon  which  the  development  of  P.n  automated  adaptive 
trainer  may  be  based.  An  F — 4 type:  aircraft  weapon  system  (pilot  and  radar 
intercept  officer)  was  selected  and  the  air-to-air  intercept  portion  of  the 
primary  mission  was  examined.  Three  adaptive  variables  were  employed: 
two  task  loading  stressors  (turbulence  and  time  pacing  of  verbal  commands); 
and  one  variable  which  changed  the  response  characteristics  of  the  vehicle 
(center  of  gravity).  For  the  two  task  loading  variables  , the  same  meas- 
urement set,  with  slight  tolerance  changes,  was  generally  adequate.  The 
center  of  gravity  variable  required  a modified  measurement  set.  While 
both  pilot  and  RIO  measures  were  developed  as  well  as  measures  for  crew 
system  performance,  the  greatest  effort  'was  devoted  to.  the  pilot  sets, 
hence  we  will  emphasize  these. 

Table  29  shows  the  suggested  performance  measures  for  task 
dimensions  grouped  according  to  aircraft  axis  systems.  An  empirical  test 
of  the  pilot  performance  measurement  candidates  for  a subset  of  the  pilot 
task  during  intercept  yielded  the  performance  measures  shown  in  Table  30. 

The  relationships  between  adaptive  variables  and  measures  were 
mixed  and  varied.  An  examination  of  the  effects  of  the  adaptive  variables  , 
one  measure  at  a time,  revealed  many  non- monotonic  functions  and  re- 
versals in  trend.  Some  adaptive  variables  caused  performance  to  improve 
on  one  task,  yet  deteriorate  on  another  task.  For  example,  increasing  the, 
command  pacing  from  15  seconds  to  5 seconds  caused  roll  average  error 
to  re  due  e for  the  60°  bank  but  inc  r eas  e for  the  30°  bank . The  r e suits  and 
discussionof  the  experiments  are  too  voluminous  to  include  here  and  the 
relationships  are  best  understood  when  read  in  the  context  of  the  report. 

In  essence,  the  recommended  measurement  sets  for  turbulence  and  com- 
mand pacing  adaptive  variables  and  for  the  center  of  gravity  adaptive 
variable  are  shown  in  Tables  3 1 and  32,  respectively. 
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TABLE  29.  MEASURE  CANDIDATES  FOR  PILOT  SYSTEM  PERFORMANCE  DURING  INTERCEPT 
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TABLE  30.  PERFORMANCE  MEASURES  (Continued) 
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TABLE  31.  MEASUREMENT  CANDIDATES  FOR  TURBULENCE 
AND  PACING  ADAPTIVE  VARIABLES 


MEASURE 

FUNCTION  OF 

TOLERANCE 

Altitude  Error 

Altitude  Command 

±100  Ft. 

Altitude  Rate 
Variability 

Steady— State 
Dive  or  Hold 

Climb/ 

Command 

±600  Ft. /Minute 

True  Airspeed  or 
IAS  or  Mach 

Steady— State 
Command 

Airspeed 

±20  knots 

or  mach 

Roll  Average 
Error 1 

Bank  Command 

±5° 

g— Error  1,2 

Bank  Command 

± . 2 g for 
± . 4 g for 

Pacing , 
Turbulence 

Turn  Rate  Error1 

Bank  Command 

±.2  deg. /sec.  for 
Pacing 

1Tliese  measures  to  be  taken  from  the  time  a command  is  first 
achieved. 


2 g— mean  shown  on  graphs ; g— error  is  the  difference  between  the 

obtained  value  and  the  desired  value*, 


( from  Vreuls  and  Obermayer , 1970) 
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TABLE  32.  MEASUREMENT  CAN DX DATE S FOR  C.G. 
AS  AN  ADAPTIVE  VARIABLE  1 


MEASURE 

FUNCTION  OF 

TOLERANCE 

Altitude  Error 

Altitude  Command 

±100  Ft. 

Altitude  Rate 
Var  i ab  i li  ty 

Steady— State  Climb/ 
Dive  or  Hold  Altitude 

±600  Ft. 

True  Airspeed, 
or  Mach 

Steady— State  Airspeed 
Command 

±20  knots  or  equivalent 
Mach 

Roll  Overshoot2 
Amplitude 

Bank  Command 

±8  degrees 

Alpha  Variabil- 
ity 2 

Any  Steady  State 
Conditions 

±1 .5  units 

2/3 

g— error 

Bank  Command 

± . 4 g 

Turn  Rate  Error2 

Bank  Command 

± . 4 deg . / sec  . 

Turn  Rate  Period2 

C.G.  Configuration 

± . 2 sec . 

Measurement  list  is  based  on  inference  from  both  studies.  It 
is  urged  that  this  list  be  considered  tentative  until  further 
analysis  of  the  effects  of  C.G.  can  be  studied. 

2 Measure  to  be  taken  from  the  time  command  is  first  achieved. 

3 g— mean  is  shown;  g— error  is  the  difference  between  the  value 
obtained  and  the  desired  value. 


(from  Vreuls  and  Obermayer  , 1970) 
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The  authors  conclude  that  the  specific  measures  for  the  pilot  (and 
the  RIO  and  total  system  which  are  not  shown  here)  may  be  used  for  adaptive 
training.  These  represent  the  best  recommendations  at  this  time.  “One  of 
the  principal  conclusions  of  this  study  is  that  measurement  for  adaptive  auto- 
mated training  in  a complex  weapon  system  trainer  is  feasible  even  though 
a number  of  fears  and  cautions  have  been  given  in  this  report.  While  the 
measurement  state-of-the-art  will  not  permit  the  design  of  such  a trainer 
without  some  empirical  testing,  current  methods  will  permit  design  of  a 
system  which  operates  on  a complex  description  of  performance  of  individual, 
team  and  system  outputs.  On  the  other  hand,  other  design  considerations  of 
automated  adaptive  trainer  design,  in  particular  the  adaptive  logic,  may  be 
somewhat  constrained  by  the  characteristics  of  the  performance  measure- 
ment which  is  now  possible  to  implement. 

In  short,  the  current  study  effort  leads  to  the  conclusion  that  the 
development  of  an  automated  adaptive  weapon  system  for  high-performance 
aircraft  is  feasible  insofar  as  design  considerations  of  performance  and 
criterion  measures  are  concerned.  Nevertheless,  the  development  of  such 
a trainer  will  require  extensive  directed  analytic  and  empirical  effort,  which 
can  now  be  identified,  to  produce  a successful  product.  11  (Vreuls  and 
Obermayer  (1970),  p.  166.) 

3*  5.  5 Research  Issues.  It  is  difficult  to  reject  the  fact  that  the  effective- 
ness of  training  is  dependent  on  the  effectiveness  of  measurement,  yet,  per- 
formance scoring  is  not  a strong  feature  in  current  training  simulators. 

The  measurement  studies  in  the  training  context  are  relatively  sparse  and 
quite  speculative  and  the  research  accomplished  to  develop  the  measurement 
capability  has  principally  yielded  only  requirements  and  feasibility  demon- 
strations in  support  of  measurement  systems  for  training. 

Thus,  the  prime  research  requirement  continues  to  be  the  develop- 
ment of  an  adequate  and  objective  measurement  capability  that  is  intrinsic 
to  classes  of  training  devices.  Much  emphasis  must  be  accorded  to  this 
development,  particularly  to  the  automated  monitoring,  evaluation  and  scor  — 
lng  in  training  devices.  Some  research  issues  of  priority  are  identified  below. 

The  design  issues  involved  in  an  automated  monitoring,  evalua- 
tion and  scoring  capability  must  be  systematically  explored  and  guidelines 
for  design  developed. 

• Work  should  continue  on  the  psychometric  analysis  of 
measurement  characteristics  including:  what  measures 

to  obtain  relative  to  classes  of  training  devices;  the  validity 
and  reliability  of  proposed  measures;  whether  variability, 
central  tendency  or  reliability  should  constitute  part  of  the 
performance  measure  (i.  e.  , for  automatic  adaptive  scoring); 
the  reliability  of  types  of  scores  as  measures  (e.  g.  , time- 
in-tolerance  scoring);  the  meaning  of  in-tolerance  scoring 
for  manual  control  (e.  g.  , i^r<tyaly-able  performance  information 
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unavailable  during  the  in-tolerance  periods?); 
and  the  like  . 


# Data  should  be  organized  on  sampling  intervals 

for  measurement.  This  includes  optimum  intervals 
for  sampling  and  averaging  performance,  how 
often  should  sampling  be  conducted,  and  when  (in 
the  maneuver  / mi s s ion  context)  is  sampling  most 
profitably  employed.  For  example,  fifty  scorings 
for  a given  aerial  maneuver  may  be  unwieldy  and 
unnecessary,  but  what  is  the  number  desired? 


, The  development  of  guidelines  for  the  organization 
of  obtained  quantitative  information  requires  a con- 
certed study  effort.  This  includes,  the  appropriate- 
ness of  single  summary  scores,  the  combination  of 
scores,  weighting,  ranking  of  scores,  etc.  To  this 
end,  a systematic  effort  should  be  devoted  to  deter- 
mining correlations  among  alternative  measures  and 
how  various  scores  are  optimally  combined. 

b.  Establishing  good  criteria  for  evaluating  the  effectiveness  of 
training  continues  to  be  a considerable  requirement.  An  intriguing  research 
issue  is  the  development  of  techniques  for  correlating  expert  opinions  with 
the  quantitative  findings  from  research.  This  suggests  the  development  of 
models  for  identifying  performance  scores  that  correlate  highly  with  sub- 
jective evaluations  of  expert  instructors. 

c.  A specific  study  of  immediate  utility  is  the  development  of 
measurement  systems  for  non-aviation  training  devices.  The  most  obvious 
candidate  is  the  tactical  team  training  device.  To  date,  the  measurement 
capability  has  not  been  an  integral  part  of  the  design  of  surface  or  subsur- 
face team  trainers,  and  performance  assessment  has,  for  the  most  part, 
been  accomplished  during  the  post-mission  critique  session  based  on  in- 
structor judgments  about  performance. 

d.  Research  is  needed  on  determining  the  formatting  requirements 
for  trainee  performance  information  displayed  or  recorded  at  the  instructor 
station.  This  is  particularly  important  in  training  situations  where  student 
to  instructor  ratios  are  high,  and  involves  the  information  requirements 
displayed  on  computer- generated  cathode  ray  tubes  and  the  information  re- 
quirements for  hard-copy  printout  records  for  critique  and  record-keeping 
purposes.  Of  concern  is  the  organization  of  the  performance  information 
(error,  monitoring  information)  and  what  classes  of  information  and  speci- 
fic data  to  display.  The  research  must  consider  CRT  page  format  require- 
ments including  the  display  of  increasingly  specific  information  by  means 

of  successive  CRT  pagination  and  formatting.  Similar  requirements  must 
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be  satisfied  for  hard-copy  printouts  of  performance  records  for  critique 
(to  be  used  both  during  and  at  completion  of  the  training  exercise)  and  for 
school  record-keeping  purposes  (normative  data)  . 

e.  A limited  effort  is  recommended  to  examine  the  extent  of  student 
involvement  in  simulator  training.  This  could  involve  determining  the  cor- 
relation of  physiological  indices  with  performance  in  simulators  to  obtain 
information  on  the  desirability  of  including  human  physiological  monitoring 
equipment  in  training  devices  (preferably,  flight  simulators).  For  example, 
in  the  flight  context,  such  equipment  may  be  used  to  obtain  information  for 
assessing  the  extent  to  which  simulction  training  induces  ’'pilot  involvement” 
during  defined  (syllabus)  simulator  sorties.  The  question  of  importance  is, 
how  successful  is  the  simulator  in  representing  the  critical  conditions  of 
flight  based  on  how  the  pilot  is  "involved,”  "aroused,  ” "mobilized”  during 
simulator  training.  The  objective  is  to  understand  the  extent  of  pilot  in- 
volvement (coping  behavior)  for  defined  simulator  tasks  as  one  .means  for 
improving  the  utilization  of  flight  simulators.  The  measures  of  involvement 
could  include: 


. Physiological  measures  (extent  of  physiological 

changes  with  such  indicants  as  heart  rate,  muscle 
tenseness,  respiratory  rate,  pupil  size,  etc.) 

• Coping  with  contingencies  (e.g.,  stereotyped 

responses,  inappropriate  response  / interference, 
frequency  of  response,  anticipation,  etc. 

f.  A research  area  of  some  significance  for  design  concerns  the 
automation  of  instructor  functions.  The  issues  concern  the  definition  of 
what  functions  are  best  automated  and  how  this  is  to  be  accomplished. 

In  essence,  the  computer  should  be  employed  to  facilitate  rational  deci- 
sions about  training.  The  values  of  computer  assisted  instruction  (CAI) 
in  simulator  training  should  be  thoroughly  explored.  The  focal  points 
involve  computer  adjustment  of  the  training  material  presented  to  the 
student  based  on  how  he  is  responding  (our  present  concept  of  automated 
adaptive  training  is  exactly  this),  and  computer  assists  for  providing  the 
instructor  with  information  needed  for  developing  optimum  strategies  for 
training  individual  students  or  teams. 


3 m 

j 

- 378  - 


O 

ERIC 


NAVTRADEYCEN  69-C-0298-1 


3.6  ADAPTIVE  TRAINING  STRATEGIES. 


A recent  innovation  for  structuring  and  controlling  training  in  simulators 
is  adaptive  simulation.  This  technique  is  essentially  a model  for  programing 
the  computer  to  achieve  strategies  for  effective  instruction. 

Adaptive  training  as  considered  here  is  restricted  to  computer- 
controlled  automation  of  the  instructor  function  in  the  synthetic  training  sys- 
tem  (i.e.  , a closed  loop  machine  - controlled  feedback  system).  The  technique 
provides  for  the  automatic  variation  of  the  task  complexity  or  difficulty  as  a 
consequence  of  how  well  the  trainee  is  performing,  that  is,  the  training  task 
automatically  becomes  more  difficult  as  the  trainee  becomes  more  skilled 
(Kelley,  1969).  Thus,  in  adaptive  training,  task  difficulty  is  automatically 
kept  appropriate  to  each  trainee 's  level  of  skill,  that  is,  the  level  of  task  dif- 
ficulty is  automatically  adjusted  to  the  trainee's  momentary  level  of  skill;  as 
he  progressively  acquires  skill,  the  task  becomes  increasingly  more  difficult. 

Adaptive  training  is  applicable  to  any  man-operated  vehicle  simulator 
as  well  as  to  other  training  situations.  The  emphasis  is  on  the  machine  or 
computer  control  in  the  development  of  a training  strategy  whereoy  the  diffi- 
culty level  of  the  training  task  is  automatically  adjusted  as  a consequence  of 
the  immediately  preceding  performance  of  the  trainee.  The  feature,  "auto- 
matic1 1 is  emphasized  in  contradis tine tion  to  the  usual  training  situation  which 
strives  to  achieve  an  optimum  progression  in  learning  based  on  an  instructor 
controlled  sequencing  strategy.  A distinction  can  also  be  made  between  auto- 
matic adaptive  sequencing  (which  is  the  topic  of  this  chapter)  and  manual 
(normal)  adaptive  sequencing  (which  involves  instructor-developed  training 
strategies  based  on  a number  of  automated  equipment  assists).  The  term, 
manual  adaptive,  may  legitimately  describe  a simulated  situation  whereby 
an  instructor  with  equipment  assists  decides  on  a strategy  and  initiates  a 
sequencing  of  training,  based  on  displayed  trainee  performance  information 
to  accommodate  each  trainee’s  capability  to  learn.  The  intent  is  to  keep  each 
trainee  progressing  at  his  own  achievement  speed  (ideally  at  the  threshold  of 
his  capability  at  any  given  time)  so  that  the  task  is  neither  too  easy  or  too 
difficult. 

These  assists  provide  instructional  flexibility  in  employing  a simulator 
for  training  and  unburden  the  instructor  from  the  more  clerical  duties  in  the 
management  of  training.  As  such,  this  describes  a semi  — automated  instruc- 
tional capability  based  on  the  following: 
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m Preprogramed,  standardized  mission  scenarios  . 

Computer  automated  measurement  and  scoring  system. 

, Computer-driven  CRT  displays  at  the  instructor  console 
for  presenting  all  relevant  performance  and  mission  in- 
formation in  defined  operating  modes. 
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. Controls  to  initiate,  continue  or  modify  the  preprogramed 
mission  events  to  achieve  a pertinent  strategy  for  each, 
s tudent . 

. Hard  copy  printouts  of  student  performance  information 
for  use  during  and  after  a mission  (critique  and  school 
records)  . 

By  these  means  the  instructor  is  able  to  formalize  a decision  logic 
tailored  to  the  trainee  based  on  a responsive  monitoring,  evaluation  and 
scoring  capability. 

3.6.1  Differences  in  Concept.  Variations  exist  in  conceptualizing 
machine -controlled  adaptive  simulation.  The  adaptive  system  developed 
by  Birmingham  ( 1959)  simplified  an  automatic  tracking  system  by  adding 
derivatives  of  error  into  the  display  signal  (quickening).  The  display 
augmentation  is  governed  by  the  adaptive  circuit  so  that  with  poor  tracking 
performance,  the  trainee  is  provided  help  via  the  display  augmenter  (see 
Figure  44).  This  help  is  lessened  as  performance  increases  until  it  is  re- 
moved as  the  preset  criterion  is  reached.  Thus,  as  performance  improves 
the  adaptive  variable  changes  with  it,  i.e.  , performance  and  the  adaptive 
variable  change  simultaneously. 

The  technique  developed  by  Kelley  (1963)  involves  averaging  a 
measurement  of  the  subject's  performance  over  a period  of  time  rather 
than  using  an  instantaneous  value  of  error  (as  did  Birmingham).  The 
average  score  is  used  to  adjust  automatically  certain  parameters  of  the 
task.  The  scoring  technique  for  use  with  continuously  distributed  error 
scores  involves  setting  an  error  threshold  which  is  an  amount  of  error 
that  will  not  produce  any  change  in  the  system.  Error  in  excess  of  the 
threshold  results  in  changes  which  makes  the  system  easier,  while  smaller 
errors  cause  the  system  to  become  more  difficult. 

Hudson's  technique  (1964)*  is  similar  to  Birmingham's  in  that  as 
the  error  is  decreased,  the  simulation  parameter  that  is  adjusted  changes 
in  some  fixed  relation  as  a function  of  improved  performance.  Thus,  the 
parameter  of  display  quickening  is  changed  in  proportion  to  an  average  or 
filtered  error  score.  In  Kelley's  technique,  error  is  held  constant  and 
system  difficulty  changes  are  adjusted  until  the  set  amount  of  error  is 
present. 

1 Als  o , Hudson,  E.M.  An  Adaptive  Tracking  Simulator.  Paper  presented 
at  the  IRE  International  Congress  on  Human  Factors  in  Electronics,  Long 
Beach,  California,  3-4  May  1962. 
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Figure  44  • Early  Adaptive  System. 

(from  Kelley.,  1969) 


(Kelley, 


The  difference  in  techniques  is  summarized  in  the  following 
19  69).  In  the  Birmingham  and  in  the  Hudson  techniques  . 


y = ax  - b 


in  the  Kelley  technique 

dy 

dt 

where: 

y = 

X = 

a and  b = 


O 


= ax  - b 

adaptive  variable 
performance  measurement 
constants  . 

- 38 1 - 

? O&Q 


NAVTRADEVCEN  69  -C  -0298  - 1 


In  the  first  equation,  as  skill  increases,  average  tracking  error  decreases, 
and  this  decrease  brings  about  an  adaptive  change  in  the  system.  The 
system  adapts  as  the  average  error  is  reduced.  In  the  second  equation, 
average  tracking  error  does  not  change  as  skill  improves.  When  the  error 
deviates  from  the  preset  standard  the  system,  changes  until  the  standard 
amount  of  error  is  again  present.  Consequently,  the  system  becomes 
more  difficult  when  performance  is  above  the  standard.  The  adaptive  vari- 
able changes  to  keep  the  average  tracking  error  constant. 

The  difference  between  the  two  basic  techniques  described  above 
resides  in  the  adaptive  logic.  For  Birmingham,  performance  is  changing 
as  the  task  is  changing;  for  Kelley,  the  derivative  of  the  performance  meas- 
ure changes  in  order  to  maintain  a constant  performance  over  a time  period 
(error  is  used  to  modify  the  system  to  make  the  task  easier  or  harder  for 
the  trainee).  The  performance  score  stays  the  same;  changes  in  the  adap- 
tive variable  serve  as  the  index  of  training.  Hudson  (1969)  utilizes  error 
to  modify  the  system  so  that  it  will  converge  quickly  on  a value  that  will 
reduce  error  to  a minimum,  i.e.  , the  system  (machine)  is  modified  to 
more  closely  match  the  human’s  performance.  Hudson  cites  the  similarity 
of  his  approach  to  that  of  sequential  analysis  where  the  number  of  experi- 
ments to  be  conducted  is  determined  during  the  course  of  experimentation 
as  a function  of  the  level  of  significance  obtained. 

3.6.2  Adaptive  Simulation  as  a Design  Option.  Perhaps  the  significant 
question  for  design  concerns  the  efficacy  of  adaptive  simulation  for  train- 
ing. Does  adaptive  sequencing  enable  the  trainee  to  achieve  desired  pro- 
ficiency levels  more  quickly  than  when  conventional  means  are  employed? 
Unfortunately,  no  empirical  evidence  exists  at  this  time  that  adaptive  simu- 
lation trains  better.  Adaptive  techniques  have  served  as  a research  tool 
and  a number  of  interesting  relationships  have  been  demonstrated.  But 
assessments  are  not  available  which  evaluate  the  effectiveness  of  adaptive 
simulation  as  a practical  training  method  for  producing  the  skills  required 
in  the  operational  environment.  The  major  difficulty  resides  in  problems 
associated  with  the  conceptualization  and  definition  of  adaptive  variables 
and  difficulties  in  the  measurement  of  performance.  These  issues  are 
discussed  subsequently  in  this  Chapter. 

Since  evidence  is  lacking  on  the  training  advantages  of  adaptive 
sequencing  in  relation  to  other  training  concepts,  just  what  are  the  per- 
ceived positive  expectations  from  its  use?  Obviously,  it  is  not  d fficult 
to  favor  adaptive  training  as  a concept  since  logically  all  training  is  (ideally) 
adaptive.  Hence,  many  theoretical  advantages  inhere  in  adaptive  simula- 
tion. As  a technique  for  presenting  information  to  the  trainee  it  has  made 
explicit  a number  of  features  that  are  intuitively  desirable.  These  include 
the  following: 
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• Formal  structure  and  control  of  training  requiring  a 
deliberate  logic  (tailoring  learning  to  the  individual 
based  on  rational  decisions  about  performance) 

# Precise  control  of  task  loading. 

• Objective  adjustment  of  the  difficulty  level  of  tasks. 

9 Structuring  the  link  between  the  trainee's  performance 
and  the  presented  task. 

# Emphasis  on  measurement  during  the  training  process. 

• Requirement  to  analyze  all  the  variables  that  can  in- 
fluence trainee  performance  and  task  difficulty*. 

# Record  of  the  training  process. 

While  these  features  are  not  peculiar  to  adaptive  simulation,  all 
are  pertinent  to  the  technique  . 


Assuming  the  efficacy  of  adaptive  simulation,  what  training  prob- 
lems are  adaptive  techniques  best  suited  to  solve?  This  is  a relevant 
question  for  design,  since  it  appears  that  not  all  training  will  benefit  from 
this  approach.  To  date,  it  has  been  applied  almost  totally  to  tracking  be- 
havior and  perceptual  motor  skills,  so  much  so  that  it  has  become  a major 
preoccupation  of  flight  simulation.  Recently,  a workshop  on  adaptive  train- 
ing 1 examined  the  question  of,  when  do  you  employ  adaptive  training  methods 
and  for  what  reasons?  Excerpts  from  this  workshop  which  highlight  vari- 
ous opinions  of  the  attendees  are  organized  below. 


* Adaptive  training  methods  are  called  for:  1)  when  the 

task  to  be  learned  is  of  sufficient  difficulty  to  require 
a significant  (?)  amount  of  time  to  achieve  mastery, 

2)  when  task  improvement  is  a linear  or  progressive 
phenomenon,  3)  when  the  training  situation  lends  itself 
to  automated  controls,  4)  when  the  subject  population 
can  be  controlled  to  the  extent  necessary,  and  5)  when 
the  training  task  bears  sufficient  resemblance  to  the 
final  behavioral  model  io  have  a reasonable  .transfer 
value  . 


"^Workshop  on  Adaptive  Training,  2,7-28  April  197  0 , Institute  of  Aviation, 
University  of  Illinois  , Urbana,  Illinois  . 
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The  availability  of  a performance  measure,  an  adaptive 
variable,  and  an  adaptive  logic  are  the  conditions  for 
the  use  of  adaptive  training,  but  of  course  this  is  not 
a reason  for  its  use.  Automated  adaptive  training  may 
be  used  for  the  purposes  of  reducing  the  need  for  human 
instructors,  standardizing  training  procedures,  making 
the  decision  function  more  objective  and  rational,  or 
achieving  greater  precision  of  control  in  presenting 
tasks  and  instructional  materials.  These  are  important 
goals  , but  they  can  be  met  by  other  methods  of  auto- 
mated, structured,  or  programed  training  that  are  not 
necessarily  adaptive.  The  purposes  that  can  be  served 
uniquely  by  adaptive  training  must  be  those  which  are 
served  by  maintaining  a continuously  closed  loop  be- 
tween the  student*s  performance  output  and  the  task- 
stimuli  input-  Nobody  truly  knows  what  clear  benefits 
are  offered  by  adaptive  training  and  only  by  adaptive 
training.  Every  potential  benefit  of  adaptive  training 
that  can  be  suggested  as  being  unique  to  this  technique 
is  properly  a hypothesis  that  requires  empirical  testing. 

Adaptive  training  methods  are  called  for  when  training 
is  already  adaptive,  but  instructor  controlled,  and  when 
it  is  desirable  for  reasons  of  cost,  standardization,  or 
(when  it  can  be  demonstrated)  training  effectiveness  to 
mechanize  the  instructor^  adaptive  function.  These 
methods  are  also  appropriate  when  it  can  be  shown  that 
training  time  is  significantly  reduced  or  that  a higher 
level  of  skill  is  reached  in  the  same  time  by  replacing 
a fixed  with  an  adaptive  training  schedule.  The  criterion 
should  be  cost-effectiveness,  broadly  interpreted  and 
applied  . 

Adaptive  training  systems  are  probably  most  useful  in 
situations  requiring  considerable  overlearning  and  high 
retention  over  time.  They  are  most  appropriate  in 
systems  where  variables  can  be  closely,  perhaps  in- 
finitely, controlled;  but  application  to  part-task/ whole - 
task  grouped  learning  patterns  may  not  be  amenable  to 
the  close  control  over  levels  of  difficulty  that  seems 
essential  to  the  technique.  The  important  problem  is 
the  relevance  of  the  adaptive  variable  used  as  a forcing 
function  (turbulence,  for  example)  to  the  task  for  which 
the  student  is  being  trained. 
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Adaptive  training  techniques  are  justifiable  whenever 
the  goal  is  to  automate  the  trailing  situation,  i.e.  , 
to  replace  the  instructor  with  a machine.  A good 
instructor  utilizes  adaptive  techniques  in  that  he 
tailors  the  training  situation  to  the  individual  trainee. 

The  primary  problem  is  to  formalize  the  decision 
logic  in  the  automated  device.  This  approach, 
hopefully,  will  allow  at  least  a standardization,  and 
perhaps  an  optimization  of  the  training  situation. 

Adaptive  training  requires  specification  of  a decision 
model  which  relates  observed  or  measured  performance 
to  task,  difficulty  or  complexity.  In  the  real-world 
model,  i.e.  , a human  instructor  working  with  a single 
trainee,  decisions  are  often  made  on  the  basis  of  the 
instructor's  subjective  evaluation  of  trainee  perfor- 
mance, and  his  decision  model  is  not  wholly  rational. 

By  removing  the  instructor,  a wholly  rational  decision 
model  can  be  substituted  for  him.  Intuitively,  we  can 
assume  that  adaptive  training  then  has  at  least  the 
potential  of  improving  upon  the  real-world  model. 

• daptive  training  is  called  for  when  the  computer  can 
substitute  for  the  instructor  and  when  the  task,  is  of 
•vjch  a high  order  of  difficulty  that  it  cannot  be  mastered 
unless  it  is  broken  down  into  component  parts.  What 
is  needed  for  implementation  is  an  analysis  of  the 
task(s)  to  be  trained  and  then  the  development  of  a 
"strategy"  for  teaching  it. 

The  development  of  adaptive  training  methods  is  linked 
to  the  goal  of  automating  many  instructor  functions  (in 
flight  simulator  training,  for  example).  One  reason 
for  doing  this  is  not  to  replace  the  instructor,  but  to 
relieve  him  of  time-consuming  routine  duties,  so  he 
can  use  his  teaching  skills  more  effectively.  it  is 
impossible  or  impractical  to  automate  some  instructor 
functions,  so  let's  not  talk  of  "getting  rid  of  him." 

Adaptive  training  methods  are  most  probably  called  j 

for  in  perceptual-motor  tasks  that  are  too  difficult  / 

for  someone  to  handle  at  the  outset,  such  as  the  task 
of  controlling  a helicopter.  But,  generally,  the  systems 
approach  must  be. taken  to  identifying  training  applica- 
tions, using  behayioral  objectives  and  task  analyses. 

If  a need  for  adaptive  training  shows  up,  then  use  it. 

But,  it  is  a xr^  stake  to  work  in  the  other  direction. 
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starting  with  the  assumption  that  adaptive  techniques 

will  be  used  and  then  trying  to  fit  them  into  the 

program. 

3.6.3  The  Elements  of  Adaptive  Simulation.  A number  of  issues  must 
be  considered  in  the  decision  to  implement  an  adaptive  simulation  capa- 
bility in  a training  device  . These  are  subsumed  under  the  key  design 
features  of  adaptive  simulation,  specifically:  the  sel  ction  of  the  adaptive 

variable(s);  the  performance  measurement  technique,  the  adaptive  logic 
for  adjusting  task  changes;  and  some  display  of  performance  information 
to  the  trainee.  Design  issues  within  each  of  those  elements  are  discussed 
next. 

3.6.3.  1 Selecting  Adaptive  Variables  . An  initial  decision  is  the  choice 
of  the  parameter  or  variable  which  will  be  manipulated  to  adjust  task  diffi- 
culty. The  task  must  be  scalable  along  some  dimension  of  graduated  diffi- 
culty, either  continuously  or  in  step  fashion.  The  adaptive  variable  selected 
must  systematically  affect  the  difficulty  of  the  task. 

A number  of  criteria  have  been  specified  for  a relevant  adap- 
tive variable.  Matheny  and  Norman  (1969),  for  example,  have  listed  the 
following  requirements  for  the  selection  of  an  adaptive  variable: 

• It  must  be  capable  of  being  described  and  quantified. 

That  is  , it  must  be  capable  of  being  measured. 

• It  must  be  related  to  task  difficulty.  That  is,  changes 
in  the  adaptive  variable  must  bring  about  changes  in 
level  of  performance. 

• It  must  be  capable  of  being  varied  in  a systematic  way. 
Although  this  seems  obvious,  in  a practical  situation, 
it  may  not  be  easy  to  do. 

• When  used  in  devices  for  training  for  transfer  to  a 
different  system,  it  should  not  inhibit  or  interfere  with 
transfer  of  training.  Parenthetically,  the  addition  of 
the  adaptive  training  feature  to  a training  task  which 
has  no  training  value  in  and  of  itself  cannot  be  expected, 
magically,  to  endow  that  task  with  positive  transfer 
characteristics. 

• It  must  be  capable  of  being  adjusted  over  a range 
commensurate  with  the  trainee’s  skill  both  in  increas- 
ing and  decreasing  difficulty.  For  example,  the  use 
of  turbulence  as  an  adaptive  variable  might  be  limited 
in  its  application  because  in  some  systems,  of  which 


NAVTRADEVCEN  69-C-0298-1 


the  helicopter  may  be  an  example  , the  task  difficulty 
for  the  beginning  student  must  be  reduced  to  some 
level  below  that  of  normal  operation  in  calm  air. 

That  is,  for  the  beginner,  the  task  may  need  to  be 
made  easier  than  it  actually  is  in  the  real  system 
for  most  efficient  training. 

• If  possible,  it  must  be  realistic  in  the  sense  that  it 
varies  in  ways  in  which  the  real  task  varies  so  that 
it  is  acceptable  to  the  trainee.  That  is  to  say  that 
it  must  not  change  the  basic  task  so  that  it  becomes 
strange  and  unreal  to  the  trainee  and  not  acceptable 
to  him  as  a training  situation. 

The  participants  in  the  recent  workshop  on  adaptive  training 1 
considered  the  proper  guidelines  for  selecting  adaptive  variables.  Excerpts 
from  the  comments  made  are  cited  below. 

m The  paramount  rule  should  be  to  select  an  adaptive 
variable  that  is  clearly  relevant  to  the  operational 
task  being  trained.  No  other  guideline  matters  if 
this  one  is  violated.  The  other  guidelines  mainly 
insure  that  the  selected  variable  is  practicable. 

That  is,  the  variations  should  be  easily  definable 
or  measurable;  the  variations  should  have  a signi- 
ficant, reliable,  and  known  effect  on  the  performance 
requirements;  and  that  effect  should  be  direct  and 
uncomplicated. 

m Adaptive  variables  should  be  selected  in  accordance 
with  these  guidelines:  its  variation  should  be  along 

dimensions  relevant  to  the  skill  to  be  taught;  its 
variation  should  be  closely  related  to  measured 
performance  so  that  it  reliably  changes  measured 
performance,  and  it  should  be  convenient  to 
implement. 

„ The  most  important  guidelines  are:  the  relevance 

of  the  forcing  function  to  the  skill  trained,  the  ease 
of  varying  task  difficulty  level,  and  the  nature  of 
difficulty  dimension  itself — that  is,  whether  the  ~ 

difficulty  is  increased  by  attention- dividing  (bells, 
lights),  a physical  challenge  (turbulence),  or  a 
design  change  (display  £ize,  control  dynamics). 


1 Op^cit.  , Workshop  on  Adaptive  Training,  University  of  Illinois. 

O 
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* Tlie  following  ingredients  should  be  included  in  any- 
set  of  guidelines  for  selecting  adaptive  variables: 

1)  sensitivity  to  the  generation  of  errors  regardless 
of  what  criteria  are  employed,  2)  possibility  and 
practicality  of  adjustment  in  terms  of  stability  and 
resemblance  to  the  real  world,  and  3)  predictability 
on  the  basis  of  some  plausible  theory. 

* Guidelines  required  for  selecting  adaptive  variables 
for  training  applications  may  differ  from  those  re- 
quired to  select  adaptive  variables  for  research 
purposes.  For  training  applications,  the  adaptive 
variables  must  be  measurable  and  must  be  related 
to  progress  toward  the  training  objective;  that  is, 
the  task  being  trained,  the  appropriate  performance 
measures,  and  so  on,  must  be  well  defined  before 
attempting  to  select  the  adaptive  variable.  For 
research  purposes  the  adaptive  variable  muct, 
again,  be  measurable;  however,  it  must  also  be 
adjustable  over  a relatively  wide  range  of  skill 
levels,  with  particularly  good  coverage  at  the 
lower  end  of  the  continuum. 

. It  seems  clear  that  the  adaptive  variable  must  do  no 
obvious  violence  to  the  training  situation. 

3,6.3.  1.  1 Classes  of  Adaptive  Variables.  Not  much  data  exist  on  the 

possible  adaptive  variables  relevant  for  classes  of  training  tasks.  There 
are,  however,  rational  choices  that  may  be  made  of  adaptive  variables 
that  seem  potentially  useful  in  producing  the  required  realism  in  the  train- 
ing task.  These  appear  to  meet  the  criteria  set  forth  earlier.  For  example, 
Kelley  and  Wargo  (1968)  have  cited  the  following  task  aspects  that  can  be 
changed  adaptively. 

m The  simulated  environment  may  be  changed 

(e.g.  , illumination,  sound  level,  temperature, 
or  simulated  gustiness  or  turbulence  may  be 
varied). 

. Stress  applied  to  the  operator  can  be  varied  (e.g.  , 
g-forces,  vibration,  oxygen  pressure,  drugs). 

. The  controlled  element  (simulated  vehicle)  can  be 
modified  (e.g.  , by  varying  the  gain  of  a stability 
augmenting  signal). 
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. The  operator's  control  can  be  varied  (e.g.  , 

by  adjusting  its  gain,  simulated  feel,  or  backlash). 

. Displays  can  be  varied  (e.g.  , by  changing  the  gain, 
lag,  amount  of  "quickening"  of  a quickened  display, 
or  prediction  span  of  a predictor  display)  . 

. Problem  generation  may  result  in  harder  or  easier 
command  trajectories  to  follow,  maneuvers  to 
perform,  etc.  This  includes  the  important  training 
application  of  progression  through  a graded  sequence 
of  tasks  or  problems  . 

. Secondary  task  loading  on  the  operator  may  be 
changed  adaptively,  adjusting,  for  example,  the 
communication  load,  or  the  fluctuations  in  engine 
instruments  requiring  monitoring  and  adjustment. 


Our  discussion  of  the  classes  of  adaptive  variables  tacitly 
assumes  the  rejoinder  that  the  surety  of  selection  awaits  more  empirical 
evidence.  Within  each  of  the  classes  of  adaptive  variable  there  is  a variety 
of  simulator  features  that  can  be  varied,  so  that  these  singly  or  in  combina- 
tion provide  a substantial  range  of  possibilities.  Since  experience  with 
adaptive  simulation  is  meager,  it  is  well  to  consider  only  a single  continuum 
of  difficulty  for  design.  At  present,  the  interactions  among  multiple  dimen- 
sions are  not  well  understood,  hence  complex  scoring  schemes  are  not 
encouraged  without  further  research.  The  classes  of  adaptive  variables 
relevant  to  training  design  are  described  next. 

a.  External  Eorcing  Function — turbulence  has  been  most 
investigated  as  an  adaptive  variable  in  flight  simulation,  employed  to  make 
the  control  task  more  difficult.  Obviously,  its  use  at  the  outset  of  training 
is  limited.  Also,  in  systems  difficult  to  control  because  of  systems  dy- 
namics, difficulty  level  cannot  be  decreased  below  the  level  set  by  the  sys- 
tem dynamics,  hence,  turbulence  may  not  be  a prime  choice.  In  defining 
the  turbulence  spectrum,  decisions  involve  variations  in  frequency,  ampli- 
tude or  both.  Bandwidth  of  the  forcing  function  appears  critical  in  that  it 
may  interact  with  rate  of  change  in  difficulty  and  the  length  of  the  interval 
during  which  performance  is  measured  (Lowes,  et  al,  1968).  Wind  velocity 
(head  wind  or  tail  wind)  appears  useful  in  that  increases  or  decreases  in 
ground  speed  influence  difficulty  level  in  terms  of  time  constraints.  Cross- 
winds  have  the  effect  of  increasing  the  workload  for  the  trainee  in  ground 
tracking  since  aircraft  headings  must  consider  this  wind  vector. 


b.  Vehicle  Characteristics--  several  types  of  difficulty  fac- 
tors have  been  identified  as  promising  for  manipulating  task  complexity. 
These  include: 
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• Control  damping  — increasing  the  moment  of 
inertia  on  the  control  motions  can  be  used  to 
make  the  task  less  difficult.  With  damping, 
the  trainee  is  not  required  to  respond  as 
quickly  as  without  damping,  hence  loss  of 
aircraft  control,  especially  during  initial 
training,  will  be  less  likely. 

• Center  of  gravity  changes — CG  shifting  re- 
sulting from  fuel  loads  may  vary  the  trainee's 
workload  in  aircraft  control. 

• Malfunctions — relevant  system  malfunctions 
are  pertinent  to  graduated  task  difficulty 
levels. 

c . Signal  Characteristics — this  includes  degradation  in  sig- 
nal characteristics  (e.g.  , noise);  the  incidence  of  signals  per  unit  of  time 
in  a mission  context  (e.g.  , density  of  ECM  signals  of  varying  priority 
classes  for  a given  time  unit),  and  simultaneity  of  signals  at  a given  time 
(e.g.  , two  or  more  high  priority  EW  signals  as  immediate  threats  in  the 
airborne  environment) 

d.  Secondary  Task  Loading  — the  secondary  task  has  been 
suggested  as  an  adaptive  variable,  i.e.  , adjusting  the  primary  task  diffi- 
culty by  secondary  task  loading.  Kelley  and  Wargo  (1967)  employed  track- 
ing error  in  the  primary  task  to  adjust  the  loading  of  a secondary  task  per- 
formed by  the  subject.  Loading  was  increased  or  decreased  as  performance 
exceeded  or  fell  below  a preset  standard.  In  this  way,  primary  task  per- 
formance was  stabilized  at  the  preset  standard  and  all  variance  in  perfor- 
mance due  to  the  independent  variable  (in  this  case,  different  sized  tracking 
displays)  was  transferred  to  the  loading  task  scores.  This  loading  task 
technique  (called  ,fc ros  s -adaptive 11  loading  yielded  a single  new  dependent 
variable  which  is  degree  of  operator  loading).  The  authors  concluded  that 
this  was  more  sensitive  to  the  independent  variable  than  were  primary  task 
scores  alone  or  in  combination  with  the  secondary  task  scores. 


However,  the  use  of  secondary  tasks  must  be  considered 
carefully  in  training  device  design,  if  only  from  the  viewpoint  of  task 
changing  or  realism.  Of  concern  is  the  issue  of  whether  secondary  tasks 
change  task  difficulty  or  actually  change  the  task.  For  flight  training , the 
variables  of  communications  and  command  pacing  are  perceived  as  realistic, 
but  they  may  add  to  task  structure  more  than  just  increasing  the  task  diffi- 
culty. In  other  words,  the  object  is  not  to  just  increase  difficulty  but  also 
to  make  the  task  more  realistic.  If  the  secondary  task  is  to  be  considered 
as  an  adaptive  variable,  then  attention  must  be  paid  to  how  the  trainee 
perceives  the  two  tasks,  the  relative  difficulty  level  of  the  two  tasks,  the 
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amount  of  trainee  activity  involved  in  performing  both  tasks  and  the  mani- 
pulation rates  for  the  secondary  task.  It  seems  best  to  argue  that  the 
secondary  task  technique  is  more  useful  as  a laboratory  research  tool 
rather  than  as  an  adaptive  variable  for  operational  training. 

e.  System  Quickening  - - Matheny  (Matheny  and  Norman,  19  69) 
has  investigated  the  efficacy  of  system  quickening  as  an  adaptive  variable. 

In  system  quickening  the  actual  output  of  the  system  is  changed  by  the 
quickening.  This  is  in  distinction  to  display  quickening,  wherein  error 
scores  are  derived  for  displayed  error  rather  than  for  system  output.  The 
investigation  thus  far,  however,  suggests  that  quickening  and  aiding  are  of 
relatively  low  priority  as  candidates  for  adaptive  variables. 

f.  Effective  Time  C ons tant- -Change s in  the  effective  time 
constant  of  the  vehicle  response  (te),  is  proposed  as  an  adaptive  variable. 
The  effective  time  constant  of  the  man-machine  system  is  a measure  of 
the  time  between  control  input  initiation  and  the  operator's  detection  of  the 
vehicle  response  to  the  control  input,  i.e.  , the  speed  with  which  the  opera- 
tor detects  the  results  of  a control  movement.  It  is  a measure  of  the  re- 
sponsiveness of  the  vehicle  to  control  movements  and  is  made  up  of  the 
speed  of  the  vehicle  response  and  the  operator's  threshold  for  detecting 
changes  resulting  from  control  inputs.  Matheny  and  Norman  (1968)  showed 
te  to  be  related  to  rate  of  learning  and  to  final  level  of  precision  of  control. 
Level  of  performance  was  related  to  the  interactive  effects  of  te  a.nd  system 
gain,  with  the  effects  of  gain  most  prominent  during  initial  learning  and  te 
more  important  in  determining  final  level  of  performance.  For  an  adaptive 
training  sequence,  optimum  gain  and  te  would  be  provided  in  initial  training. 
As  training  progressed,  gain  and  te  would  be  adjusted  to  achieve  the  task 
difficulty  progression. 

g.  Changes  in  Control  Or der - -Hudson  (1964)  suggested  that 
changes  in  control  order  during  training  is  an  important  issue;  optimum 
learning  may  occur  when  the  practice  task  involves  moderate  or  average 
level  of  difficulty  regardless  of  the  level  of  difficulty  of  the  criterion  task. 
Level  of  training  achieved  is  largely  a function  of  level  of  difficulty  during 
practice. 

3. 6. 3. 2 Measuring  Performance.  Adaptive  measurement  techniqu  ; is 

relatively  new  and  not  much  data  for  design  are  available.  However*  ob- 
taining relevant  and  reliable  performance  measures  is  as  critical  as  is  the 
selection  of  the  adaptive  variables  for  training,  so  much  so  that  the  success 
of  adaptive  training  is  predicated  on  adequate  measurement.  The  research 
to  date  has  focused  on  tracking  tasks  using  traditional  performance  indices 
(absolute  error,  time-on- tar  get)  as  a performance  criterion  to  adjust  an 
adaptive  variable  affecting  the  task. 
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Kelley  and  Wargo  (1968)  recommend  that  performance  in  vehicle 
control  be  measured  along  several  dimensions  with  a separate  standard  or 
criterion  of  performance  established  for  each  dimension.  Thus,  perfor- 
mance should  be  measured  in  each  dimension,  scored  separately  and  com- 
bined into  an  overall  score.  The  authors  have  provided  an  example  for 
satisfying  measurement  requirements  for  device  2B24,  Synthetic  Flight 
Training  System.  Possible  performance  measurements  for  the  training 
device  begin  with  basic  vehicle  control  parameter s --air speed , altitude, 
and  attitude  during  steady  state  flight.  Error  tolerances  for  each  param- 
eter must  be  established  empirically  for  various  flight  conditions. 
Measurement  should  also  consider  task  components  such  as  vehicle  status 
monitoring,  navigation  and  communication  even  though  these  are  more 
difficult  to  measure  effectively.  The  scoring  of  flight  path  control  during 
maneuvers  can  be  achieved  by  computing  repetitively  the  response  of  an 
"ideal"  pilot  which  is  compared  to  the  trainee  response  on  the  basis  of 
vehicle  output  parameters  since  there  is  a wide  range  of  operator  control 
manipulations  that  will  result  in  the  desired  vehicle  response.  Instantane- 
ous performance  should  be  measured  in  terms  of  basic  vehicle  state  param- 
eters . The  parameters  suggested  are:  three  coordinates  of  position;  air 

speed;  heading;  rate  of  climb/descent;  and  pitch  (roll  and  cross-track 
velocity  could  be  used  if  found  to  be  important).  A nominal  value  should 
be  computed  for  each  scoring  parameter  via  a computer-based  model 
representing  the  desired  vehicle  state  at  each  instant,  and  recomputed 
periodically  based  on  new'  real-time  parameters.  Error  tolerances  about 
the  nominal  values  for  each  scoring  parameter  should  be  established  (e.g.  , 
four  categories  ranging  from  "within  tolerance"  or  "far  out-of- tolerance ”)  . 
An  accumulating  error  score  should  be  kept  for  each  scoring  parameter. 
Scores  not  in  tolerance  should  be  weighted  (incremental).  Performance 
on  all  parameters  together  should  be  assessed  by  logical  rules  (adaptive 
logic)  to  determine  if  adjustments  in  task  difficulty  are  warranted.  (In 
determining  the  adaptive  logic,  tolerance  limit  settings  and  logic  rules 
require  examination  of  score  distributions  and  intercorrelations  on  tasks 
performed  by  skilled  pilots). 


Kelley  has  indicated,  however,  that  the  measurement  approach 
as  presently  planned  for  device  2324  is  of  doubtful  reliability  and  does  not 
appear  to  provide  effective  adaptive  task  control.  This  shortcoming,  in 
our  opinion,  is  applicable  at  present  to  all  adaptive  training. 
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3.  6. 3. 3 Adaptive  Logic.  A training  system  is  adaptive  to  the  extent 
that  task  difficulty  level  is  an  automatic  function  of  measured  performance 
(no  instructor  intervention).  The  adaptive  change  occurs  as  a result  of 
the  difference  between  a preset  performance  standard  and  the  latest  meas- 
urement of  performance.  Thus,  the  adaptive  logic  links  the  adaptive  vari- 
ables and  performance  measurements.  These  are  the  sets  of  rules  (in  the 
form  of  equations  or  adjustment  rules  describing  the  relationship  between 
measurement  and  the  adaptive  variable)  for  adjusting  task  difficulty  as  a , 
function  of  trainee  performance. 
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Kelley  (Kelley  and  Wargo,  1968;  Kelley,  1969)  describes 
typical  time  Histories  of  an  adaptive  variable  where  rate  of  change  is  a 
function  of  different  adaptive  equations.  Figure  45  shows  these  adaptive 
learning  curves.  Curve  A shows  a continuously  variable  rate  of  change. 
Whenever  the  system  error  is  less  than  the  criterion,  the  trainee  is  per- 
forming above  standard  and  the  adaptive  score  increases,  thus  the  task  is 
made  more  difficult.  When  the  error  is  above  the  standard,  the  adaptive 
score  decreases  and  the  task  is  made  easier.  Curve  B shows  adaptation 
limited  to  one  of  two  fixed  rates.  When  performance  is  better  than  the 
criterion,  system  difficulty  is  increased  at  a fixed  rate;  when  performance 
is  less  than  the  criterion,  system  difficulty  is  decreased  at  another  fixed 
rate.  Curve  C is  similar  to  curve  B,  except  that  there  is  a central  T,dead- 
band,  11  or  zone  in  which  the  system  does  not  change  at  all.  The  level 
portions  of  the  curve  indicate  those  periods  in  which  the  subject  is  perform- 
ing at  close  enough  to  standard  that  no  change  in  system  difficulty  is  taking 
place.  Curve  D shows  the  course  of  learning  when  a system  changes  by 
discrete  units  at  fixed  intervals  of  time.  This  particular  system  also  has 
a ’'deadband,  11  so  that  when  measured  performance  is  close  to  standard,  no 
change  takes  place  in  the  system.  The  long  segments  of  the  curve  occur 
where  the  system  stays  the  same  for  two  time  periods  instead  of  just  one. 
These  indicate  periods  in  which  performance  was  very  close  to  standard. 

All  of  the  curves  indicate  successful  techniques  for  implementing  adaptive 
training. 

Equations  for  adaptive  logic  pertinent  to  various  task  require- 
ments have  been  recommended  by  Kelley  and  Wargo  (1968).  Formulae  for 
achieving  an  instantaneous  level  and  an  average  level  of  adaptive  variables 
are  shown  below. 


Equations  for  Instantaneous  Level 

of  Adaptive  Variable  Application 


( 1)  D = K[(  1 -A)  R - AW]  + ^initial  * 

D = the  adaptive  variable,  a 

higher  score  representing 
a more  difficult  condition 

K = the  sensitivity  coefficient 
determining  the  size  of  in- 
crements and  decrements 
in  D 

R = number  of  previous  right 
re  sponses 


Tasks  in  which  measured  re- 
sponses are  discrete  in  time 
and  dichotomous  or  dichotomized 
in  extent.  ,fD,f  is  incremented 
each  trial  by  one  amount  or 
decremented  by  another.  The 
relative  size  of  an  increment  vs. 
a decrement  determines  the  level 
adapted  to.  A decrement  equal 
to  9 times  the  increment  will 
cause  adaptation  to  a level 
where  1 response  in  10  is  decre- 
mented,, i.e.  , the  90  percent 
level. 
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A 


A.  = K(X  — XT~). 

B.  22-  — KIt  when  X > XT;  — K,,  when  X < Xt. 

C.  K,,  when  X > X,;  = 0.  wh  enXI'>X>  X,; 
= Kt  when  X < X,. 

D.  y t+i  = (y*  KX'),  when  X > Xz;  — y<,  when 

Xx>  X > Xt;  = (y«  — /CAT)  W/2£?7I 

x<xt: 

(i=  /rrVi/  number .) 


Figure  45  . Adaptive  Learning  Curve 
(from  Kelley^  1969) 
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Equations  for  Instantaneous  Level 

of  Adaptive  Variable  (Continued)  Application 

W = number  of  previous  wrong 
responses 

A = the  desired  proportion  of 

right  responses  at  threshold 


The  desired  threshold  in  percent  is  equal  to  100A. 


(2) 


C 

C 


c)dt  + ^initial* 


the  instantaneous  adaptive 
score  adjusting  task  difficulty, 
a higher  scoring  correspond- 
ing to  better  performance  and 
a more  difficult  task 


Tasks  in  which  measured  re- 
sponse is  continuous  in  time 
and  in  extent,  as  in  tracking 
and  manual  control.  Adaptation 
rate  is  proportional  to  the  dif- 
ference between  desired  and 
actual  level  of  response . 


K = the  sensitivity  coefficient 

governing  the  rate  of  increase 
or  decrease  of  C 


c = tracking  error,  however,  that 
may  be  defined.  It  may  refer 
to  mean  absolute  error  in  a 
single  axis,  or  to  multi  axis 
vector  error,  to  proportion 
of  time  on  target,  etc. 

CL=present  error  threshold;  when 
c = Cp,  C does  not  change  and 
the  task  remains  fixed 
n 

^ ~ -^initial  + ^ ~ 9 

i = l 

D = the  adaptive  score  at  the  nth 
measurement 

K = the  sensitivity  coefficient 
determining  the  amount  of 
change  in  D per  unit  differ- 
ence between  X and  X-j^ 


Tasks  in  which  measured  re- 
sponses are  discrete  in  time 
and  continuous  in  extent,  the 
equivalent  of  (2)  for  discrete 
responses.  Adaptive  level  is 
incremented  or  decremented 
each  measured  response  in  pro- 
portion to  the  difference  between 
desired  and  actual  level  of 
response. 


(3) 
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Application 

Xj=  each  of  the  n measurements, 
whether  they  be  individual 
scores  or  a running  average 

Xj^=a  constant,  the  preset  error 

threshold  or  desired  value  of  X 


Equations  for  Instantaneous  Level 
of  Adaptive  Variable  (Continued) 


Equation*  for  Average  I ev*.l  of 
Adaptive  Variation 


Oa 

and 

3*) 


D 


1 

n-r 


i “ r * I 


D 


»• 


©I®  the  ith  adaptive  score,  D, 
figured  as  of  Equation  (I) 
or  II),  and 


r = the  number  of  settling  trials, 
after  which  averaging  of  the 
adaptive  score  begins,  and 


Average  of  the  adaptive  variable 
D far  discrete  responses  (Equa- 
tions (11  and  (I)),  aver  so  aver- 
aging series  of  n-r  responses; 
taken  after  a series  of  r settling 
responses  are  made  to  bring  D 
to  approximately  the  appropriate 
level. 


» = the  total  number  of  trials  or 
scores,  including  the  settling 
senes,  »i  - »r 


C = the  time  averaged  value  o#  C 
in  Equation  2 

f'|5-  start  of  averaging  period  |e«4 
of  settling  time),  and 


Average  of  the  adaptive  variable 
C for  continuous  response  (Sqsa- 
non  (3l|  over  an  averaging  trial 
of  T>  * V I seconds  duration; 
taken  after  a settling  per -ed  of 
T|  second#  is  made  to  bring  <2 
to  approxioaately  the  appropriate 
level. 


T&*  total  length  of  the  trial  in, 
eluding  the  settling  period 
Of  length  T 1 
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The  error  standard  must  be  carefully  established.  The  trainee’s 
actual  moment -to-moment  performance  is  compared  against  this  criterion 
and  task  difficulty  level  is  adjusted  as  a result  of  this  comparison.  Obvi- 
ously. there  is  an  optimum  standard;  less  than  this  will  degrade  training 
efficiency.  When  the  standard  is  too  tight  the  error  tolerance  is  small 
and  the  adaptive  system  will  automatically  remain  less  difficult  to  the 
trainee;  when  the  standard  is  loo  loose,  the  trainee  will  progress  more 
rapidly  to  a more  difficult  task  configuration. 

Another  mistake  that  can  be  made  in  mechanizing  the  adaptive 
logic  is  failing  to  achieve  the  optimum  in  length  of  the  performance;  sampling 
periods.  When  the  sampling  period  is  too  short,  exaggerated  changes  in 
difficulty  level  may  occut  and  task  difficulty  may  exceed  the  trainee's 
momentary  skill  level.  Thus,  the  design  decision  must  consider  carefully 
the  implications  of  slow  varying  vs.  quick  varying  adaptive  tasks,  particu- 
larly. if  large  fluctuations  in  trainee  performance  are  expected,  since 
these  may  force  the  system  to  adjust  difficulty  level  excessively.  In  this 
sense , slow  varying  adaptation  represents  a more  reasonable  seine  tie,, 
than  does  quick-varying  adaptation. 

Display  of  Performance  Information.  Mention  should  be  made 
of  the  feedback  of  performance  information  to  the  trainee  *«nee  motivation 
to  perform  in  synthetic  training  systems  is  an  issue  of  <$ome  concern  for 
design,  to  essence,  the  kind  of  adaptive  simulations  described  earlier 
does  not  provide  the  trainee  with  any  clear  evidence  of  progress.  Task 
t Insure  is  not  achieved  by  site  trainee  since  as  skill  increases  so  does 
task  difficulty.  The  experience  of  improvement  is  blunted  (insufficient 
task-inherent  knowledge  of  results)  and,  by  definition,  complete  task 
mastery  cannot  be  achieved. 

The  consensus  is  that  supplementary  knowledge  of  results 
should  be  displayed,  even  though  artificial  and  not  directly  related  to  the 
criterion  task,  Some  digital  readout,  of  the  momentary  status  of  the  a *»p- 
ttve  variables  seems  sufficient.  Device  Ji84,  for  example,  will  employ 
a meter  which  gives  the  level  of  difficulty  of  »ne  system  at  each  ianm  «. 

A two* part  score  is  proposed.  The  first  part  (2-3  digits)  #^e«tifWs  t.b? 
mission  or  the  problem  installed;  the  second  part  (1*2  digits)  identifies 
the  level  of  problem  difficulty  as  determined  by  the  intent! :y  of  the  adaptive 
va  riable , 

Indicator  lights  may  be  used  to  depict  out *of *wl#r#«ce  condi* 
tions  as  they  emerge.  Augmented  feedback  may  also  be  provided  directly 
on  a primary  display,  for  example,  in  the  form  of  an  alpha -numeric  mes* 
sage  alerting  the  trainee  to  a specific  out -of -tolerance  error. 


<m  ... 
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The  issue  of  whether  knowledge  of  performance  information 
should  be  displayed  continuously,  teriodically  or  event  oriented,  and  di- 
rectly or  i lirectly  cannot  be  resoived  at  this  time.  It  is,  nevertheless, 
of  importance  to  design. 

3.8.4  Adaptive-  Sequencing  in  the  Synthetic  Flight  Training  System 
(Device  2R14),  The  first  atterrot  to  include  adaptive  simulation  in  a flight 
trainer  is  in  Device  2B24.  Thss  aeviee,  designed  within  the  Army's  con- 
cept of  a Synthetic  Flight  Training  System  (SFTS),  consists  of  four  UH-1 
helicopter  simulators  and  associated  motion  platforms,  a general  purpose 
digital  computer  and  an  instructor  station  (Caro,  1069;  NTDC,  1968).  A 
design  goal  has  been  to  provide  * gh  fidelity  simulation  of  the  helicopter 
pilot's  task  and  to  treat  the  instructor -simulator  combination  as  a pilot 
production  subsystem.  The  attempt  has  been  to  automate  those  instructor 
functions  that  would  enhance  system  operation.  Although  Device  2824  has 
m number  of  automatoti  instructional  assists,  iff  r.o$  primarily  an  adaptive 
training  system.  At  present,  the  adaptive  simulation  U * relatively  small 
part  of  the  system,  but  one  that  perhaps,  over  time*,  will  grow. 

The  selection  of  adaptive  variables  began  with  the  Army -imposed 
restriction  that  the  variables  have  a plausible  relationship  to  flying  so  as 
to  encourage  task  acceptance  by  the  trainees.  At  present,  four  adaptive 
variables  have  been  selected  for  implementation.  These  are: 

, turbulence  (either  longitudinal  axis  or  roll  axil 
emphasis) 

„ control  damping  (increasing  the  moment  af  inertia 
on  the  control  motions- -i.e. , airframe  response) 

, horizontal  wind  (headwind  or  tailwind  to  modify 
ground  speed) 

, cress wind 


Measurement  will  consider  the  important  parameters  of  aircraft 
control  and  performance  tolerances  have  been  set  based  on  best  estimates 
guided  by  experience.  Table  S)  lists  the  performance  tolerances  for 
specified  performance  parameters.  Error  scoring  for  the  purpose  of 
adjusting  task  difficulty  will  be  in  percent  time  out-of-tolerance  for  each 
scored  parameter.  As  it  now  stands,  the  scores  will  be  treated  separately 
for  each  parameter  rather  than  being,  combined  into  a single  score.  To 
increase  task  difficulty,  performance  must  be  within  tolerance  for  a desig- 
nated percent  of  the  sample  lime  on  all  parameters  being  used  in  a training 
exercise. 
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TABLE  33.  PERFORMANCE  TOLERANCES  'DEVICE  2B24) 


Performance  Parameter 

Acceptable  Tolerances 

Altitude 

_4l00  feet 

Airspeed 

4-10  knots 

Heading 

+5° 

11 

j Vertical  Speed 

+200  fpm 

Rate  of  Turn 

+1®/ sec 

j Ball  Position 

it/4  ball 

1 Pitch  Attitude 

44®  j 

{Roll  Attitude 

i 

wmn  | 

Course  Deviation 

4 1 dot  J 

j 

Clide  Slope  Deviation 

4 l dot  1 

-SOT*  1 

Torque  Pressure 

4 4 psi 

-m*w* 

The  #Mir#  task  interval  has  been  arbitrarily  divided  into  10-second 
intervals*  i.e.,  every  10  seconds.  tim*  out-of-tolerance  and  error  rate  is 
examined  ©n  each  parameter.  In  the  initial  programing,  error  rate  may  he 
set  at  10  percent.  Task  difficulty  will  not  change  if  performance  is  within 
tolerance  90%  of  the  time  during  the  sample  period.  If  performance  is 
acceptable  less  than  90%  of  the  time*  task  difficulty  will  decrease:  task 
difficulty  will  increase  if  acceptable  performance  exceeds  these  limits. 

The  criterion  function  is  reset  to  aero  whenever  the  difficulty  level  changes. 
The  magnitude  of  the  adjustment  in  task  difficulty  will  he  a function  of  the 
particular  adaptive  variable  being  employed  in  each  problem. 

Each  trainee  will  receive  knowledge  of  results  of  progress  via  a 
two* pari  digital  readout.  The  first  part  indicates  the  problem  being  prac- 
ticed Cihis  would  indicate  level  of  difficulty  or  stage  of  training  if  exercises 
are  graduated  in  difficulty):  the  second  part  indicates  the  level  of  problem 
difficulty  as  determined  by  the  intensity  of  the  adaptive  variable.  Thus*  a 
score  of  18.  $0  indicates  that  problem  18  is  in  effect  with  a moderate  intensity 
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level  for  the  adaptive  variable  being  employed  (i.e.  , entry  difficulty  level 
of  3;  exit  criterion  difficulty  level  of  9). 

It  is  emphasized  that  the  above  describes  initial  programing  re- 
quirements since  the  device  is  not  expected  to  be  on-line  prior  to  1971. 

Thus,  adjustments  will  be  made  and  concepts  may  be  revised  through  a 
series  of  successive  approximations  until  the  optimum  values  are  achieved 
as  experience  with  the  on-line  device  accumulates. 

The  problems  besetting  the  SFTS  in  implementing  adaptive  se- 
quencing have  been  considered  by  the  recent  adaptive  training  workshop 
at  the  University  of  Illinois.  * Excerpts  from  this  deliberation  are  described 
below. 


• The  major  problem  is  in  the  performance  meas- 
ured* The  validity  and  the  reliability  of  the 
proposed  measures  can  be  questioned.  Research 
is  recommended  using  the  SFTS,  on  the  perfor- 
mance measurement  problem  before  very  much 
reliance  is  placed  on  the  adaptive  training  fea- 
ture of  the  system.  For  example,  percent  of 
time-out-of-tolerance  may  be  a useful  meas- 
ure for  some  of  the  training  but  i^s  usefulness 

is  a function  of  the  validity  of  the«tolerances 
employed.  > 

• The  10-second  interval  of  performance  measurement 
for  determining  adaptive  changes  will  probably  be  too 
short  . The  available  empirical  evidence  on  adaptive 
training  seems  to  favor  the  "slowly  adapting"  system, 
and  so  does  theory.  Moreover*  on  the  grounds  of 
psychophysics  alone,  there  are  few  aspects  of  human 
performance  that  can  be  measured  reliably  in  10 
seconds;  and  if  the  performance  measurement  is 
unreliable,  then  of  course  the  adaptive  "logic"  is 
reduced  to  a largely  random  process. 

• The  time -out -of- tolerance  measures  of  performance 
may  have  to  be  better  defined.  It  is  possible  that 
much  information  of  potential  value  may  be  lost 
during  the  "in- tolerance"  performance.  In  addition, 
the  very  complexity  of  the  SFTS  model  may  obscure 
some  of  the  important  factors  involved  in  the  adap- 
tive training  process. 

*Op.  cit. . Workshop  on  Adaptive  Training*  University  of  Illinois, 
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• The  planned  appraoch  that  calls  for  the  trainee  to 
practice  control  on  one  axis,  then  on  two  axes,  and 
so  on,  will  lead  to  difficulties  in  transfering  from 
one  task  to  another  in  the  sequence.  The  freezing 
of  any  axis  or  set  of  axes  may  cause  the  trainee  to 
adapt  to  response  modes  that  are  incompatible  with 
response  modes  required  when  the  axis  or  axes  are 
unfrozen.  This  situation  could  result  in  significant 
negative,  or  at  least  neutral,  transfer  from  oce  task 
to  the  next. 

v A good  start  has  been  made  on  the  choice  of  adaptive 
variables.  Of  course,  the  adaptive  logic  requires 
empirical  study  with  a great  deal  of  attention  given 
to  the  standards  employed.  The  overriding  problem 
in  the  SFTS  implementation,  however,  is  that  of 
performance  measurement.  There  is  doubt  that 
the  measurement  approach  presently  planned  is 
reliable,  or  that  it  will  result  in  effective  adaptive 
task  control. 

• Early  in  the  SFTS  implementation  a great  deal  of 
refinement  is  going  to  have  to  take  place  in  the  exit 
criteria  and  in  the  way  a student  is  tracked  through 
his  training.  It  is  anticipated  that  the  1 to  9 scheme 
for  scaling  difficulty  will  have  to  be  revised  and  that 
a system  for  accumulating  milestone  "misses"  will 
have  to  be  incorporated.  Also,  student  motivation 
will  probably  not  be  handled  very  well  by  the  current 
scheme . 

The  same  workshop  or  adaptive  training  cited  above  considered 
high  priority  research  requirements  fjr  the  SFTS.  Excerpts  of  the  pro- 
posals are  listed  bvlow. 

• The  highest  priority  research  should  involve  the  improve- 
ment of  the  adaptive  features  of  the  SFTS  itself.  Although 
primary  attention  should  go  to  the  problem  of  performance 
measurement,  both  adaptive  logic  and  adaptive  variables 
should  be  studied.  At  the  next  level  of  priority,  the 
SFTS  should  be  used  to  explore  a broad  spectrum  of 
different  and  more  sophisticated  areas  relating  to  adap- 
tive training.  An  example  would  be  diagnostic  program- 
ing; measuring  special  complex  aspects  of  performance 
(pilot-related  oscillation,  engine  instrument  monitoring) 
and  using  diagnostic  measurements  for  appropriate 
instruction  to  remedy  the  faults  or  problems  diagnosed. 
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. As  a matter  of  strategy,  the  first  objective  should  be  to 
accomplish  that  research  which  will  most  enhance  the 
validity  of  the  SFTS  for  its  assigned  mission:  the 

training  of  helicopter  pilots.  The  directors  of  the  SFTS 
program  should  resist  any  immediate  temptation  to  use 
the  SFTS  as  a general  research  tool  for  exploring 
theoretical  issues.  The  technology  of  adaptive  training 
would  be  better  served  by  a mission-oriented  research 
program.  When  it  has  been  demonstrated  that  the 
SFTS  works--that  it  is  an  effective  and  practical  train- 
ing system--its  versatility  can  be  turned  to  the  purposes 
of  basic  research.  In  other  words,  SFTS  priority  should 
go  to  the  need  to  solve  its  own  problems.  After  all,  if 
the  SFTS  fails,  adaptive  training  as  an  instructional 
technique  fails,  for  it  is  billed  (intentionally  or  not)  as 
the  epitome  of  adaptive  training. 

• Four  research  areas  are  of  a high  priority.  First,  the 
performance  measures  and  performance  criteria  required 
to  effect  the  adaptive  training  feature  must  be  developed 
and  validated.  Second,  a transfer-of-training  study  is 
needed  to  compare  non -automated,  instructor- managed 
training  with  the  SFTS  adaptive  training  concepts. 

Third,  alternative  adaptive  variables,  variables  in 
addition  to  turbulence  and  control  damping,  should  be 
investigated.  A particularly  interesting  possibility 
would  be  the  use  of  task  structuring  by  means  of  various 
degrees  of  task  loading  via  malfunction  insertion. 

Finally,  research  is  needed  to  determine  the  optimum 
relationship  between  the  instructor  and  students  in  the 
SFTS.  How  can  the  instructor  best  be  kept  informed 
of  the  progress  of  the  students?  Is  there  an  optimum 
instructor-student  ratio? 

, An  experiment  on  the  most  appropriate  ways  of  pre- 
senting feedback  to  the  student  regarding  levels  of 
difficulty  is  suggested.  The  following  methods  of 
presenting  feedback  could  be  evaluated:  warning 

the  student  concurrently  with  a change:  notifying 
the  student  concurrently  with  a change:  notifying 
the  student  only  after  a change  in  difficulty  level 
has  occurred:  and  providing  him  no  feedback  at  all. 

A second  series  of  experiments  might  compare  the 
simultaneous  variation  of  adaptive  variables  to  their 
one-at-a-time  variation.  Finally,  the  use  of  the 
SFTS  as  a predictor  of  subsequent  pilot  performance 
should  be  investigated.  Measures  of  performance, 
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or  levels  of  difficulty  attained,  during  early  trials 
may  be  highly  correlated  with  measures  of  final 
pe  rformance. 

. The  initiation  of  research  studies  is  recommended 
that  would  provide  the  following:  proof  of  the  validity 

of  the  adaptive  techniques,  guidelines  for  the  selec- 
tion of  candidate  adaptive  variables , value  of  simula- 
tion versus  flight  methods  for  training;  the  contribution 
of  major  hardware  subsystems  in  simulation;  and  psy- 
chological and  engineering  data  that  would  be  available 
for  study  by  outside  groups. 

• The  SFTS  would  be  a good  vehicle  in  which  to  explore 
how  adaptive  logic  can  be  applied  to  the  progression 
or  regression  from  unit  to  unit  within  a task  element. 

Such  an  effort  could  help  answer  some  of  the  questions 
about  measurement  needs,  the  criteria  to  be  used,  the 
computer  programing,  and  the  hardware  requirements 
in  moving  from  automated  demonstration  to  guided 
practice  to  adaptive  practice,  and  back  again. 

. Parametric  studies  of  all  kinds  should  be  conducted. 

For  example,  what  are  the  optimal  rates  of  adaptation? 

Our  greatest  need  right  now  is  for  numbers.  A related, 
but  independent,  problem  concerns  the  efficacy  of 
"adaptive"  techniques  relative  to  "non-adaptive"  tech- 
niques. The  SFTS  could  be  the  vehicle  with  which  to 
conduct  a full-blown  transfer  study  involving  real  pilot 
trainees  and  real  helicopters. 

• The  highest  priority  should  be  given  to  experimentation 

involving  the  following  variables:  task  difficulty  incre- 

ments and  decrements,  length  of  the  performance  sampling 
periods,  types  of  performance  measures,  and  character- 
istics of  the  adaptive  logic. 

3.6.5  Research  Issues.  Adaptive  sequencing  as  a technique  for  program- 
ing the  computer  is  a new  approach  for  training  device  design  and  not  much 
data  are  available  as  guides  for  design.  It  is  a "wide  open"  area  for  re- 
search involving  many  problems  and  issues  to  be  resolved  before  design 
guidelines  can  be  established.  At  this  time,  we  are  unable  to  use  the 
adaptive  technique  efficiently  in  training  without  research  and  application 
in  a number  of  areas.  The  most  prominent  research  needs  reside  in 
selecting  adaptive  variables,  in  defining  the  measurement  requirements 
and  in  specifying  the  adaptive  logic  to  be  employed. 
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The  issue  of  the  task  difficulty  continuum  for  arranging  a series 
of  tasks  is  subject  to  scrutiny.  The  formal  work  to  date  has  been  con- 
cerned primarily  with  closed-loop  tracking  tasks.  While  aaaptive  sequenc- 
ing in  terms  of  difficulty  gradations  has  utility  for  flight  vehicle  simulation 
there  are  arguments  favoring  the  progression  of  behavior  in  training  in 
addition  to  the  variation  in  task  difficulty  (easy  to  difficult  task  presentation), 
i.e.  , the  ordering  of  materials  is  as  important  as  a difficulty  progression 
in  learning.  Certain  job  situations  involve  the  acquisition  of  a set  of  skills 
before  additional  skills  can  be  mastered,  and  these  may  be  no  more  diffi- 
cult than  the  preceding  set.  This  is  particularly  so  in  cognitive  tasks  and 
in  verbal  learning.  The  familiar  part-task  to  whole-task  learning  progres- 
sion is  meaningful  here.  The  idea  of  the  task  remaining  the  same  does  not 
effectively  handle  situations  where  subtasks  are  progressively  added  to  the 
learning  situation  so  that  the  amount  of  information  learned  increases  as 
a prelude  to  handling  other  aspects  of  the  job. 

The  whole  issue  of  performance  measurement  needs  resolution. 

The  validity  and  reliability  of  measures  for  adaptive  sequencing  must  be 
systematically  explored.  For  vehicle  systems,  research  to  date  has  ex- 
amined adaptation  along  a single  continuum.  Data  are  not  available  on 
systems  adapting  independently  along  several  dimensions.  Research  is 
needed  on  combining  several  variables  into  a single  score  of  performance 
in  which  individual  criteria  and  weightings  of  the  components  of  the  com- 
bined measurement  are  established. 

Much  work  is  also  required  to  establish  uata  on  the  length  of  per- 
formance sampling  periods  and  on  criterion  limits.  Loose  vs.  tight  per- 
formance tolerances  is  an  issue  of  some  concern  in  terms  of  slow- varying 
vs.  quick- varying  adaptive  tasks.  Changes  in  the  adaptive  variable  must 
not  yield  erratic  changes  in  the  levels  of  difficulty. 

The  specific  researchable  issues  are  many  and  far-reaching  con- 
cerning the  attempt  to  use  adaptive  sequencing  to  advantage  in  the  design 
of  training  devices.  Those  that  appear  to  have  priority  relative  to  the 
application  of  adaptive  simulation  to  practical  training  problems  are  iden- 
tified next. 


a.  An  initial  requirement  is  to  accumulate  insights  and  data  on 
the  values  of  adaptive  sequencing  for  classes  of  training  devices.  A legiti- 
mate question  is,  does  adaptive  training  provide  better  training  than  does 
more  conventional  approaches  (in  terms  of  costs/ time,  transfer  of  training, 
retention,  acceptance  motivation,  skill  levels /abilities  of  trainees,  etc.). 
What  are  the  advantages  for  simulator  design?  What  principles  define  adap- 
tive training?  What  constraints  are  associated  with  adaptive  sequencing? 
What  task  situations  are  most  amenable  to  this  approach  (i.e.  , what  classes 
of  training  device  would  benefit  most)?  Is  it  advantageous  to  use  adaptive 
and  conventional  sequencing  in  day-to-day  operational  training?  These  are 
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pertinent  considerations.  For  example , in  Kelley  *s  technique,  performance 
is  held  constant  while  the  task  changes  in  difficulty.  In  effect,  the  trainee 
lacks  the  opportunity  for  complete  task  mastery.  What  effect  does  this  have 
on  training?  Thus,  exploratory  study  is  needed  to  put  into  perspective  the 
reasonable  expectations  from  the  use  of  adaptive  training  as  a prelude  to 
shaping  a meaningful  and  consistent  research  effort. 

b.  The  definition  and  exploration  of  the  relevant  classes  of  adaptive 
variables  should  continue  as  a priority  effort.  This  should  involve  analyses 
of  the  variables  that  will  influence  trainee  performance  or  task  difficulty. 

In  addition  to  th,e  variables  listed  earlier  in  this  Chapter,  a number  of  others 
seem  relevant,  for  example,  alerting  cues.  Since  much  of  the  available 
research  has  been  devoted  to  perceptual-motor  adaptive  models,  research 
is  needed  on  the  use  of  adaptive  concepts  and  the  definition  of  adaptive 
variables  for  training  situations  quite  different  from  aircraft  control  situa- 
tions, for  example,  cognitive  skills,  verbal  learning,  and  tactical  decision- 
making behavior . 

Assessing  and  equating  difficulty  levels  between  adaptive  vari- 
ables ie  also  recommended.  This  would  involve  explicit  descriptions  of 
difficulty  as  a function  of  performance.  So  far,  two  opposed  techniques 
have  formulated  the  relationship  between  difficulty  and  performance.  For 
Kelley,  performance  is  held  constant  while  the  task  changes  in  difficulty; 
for  Birmingham  and  for  Hudson,  task  difficulty  increases  as  performance 
increases . 


c.  The  performance  measurement  requirements  are  crucial  since 

adaptive  training  depends  on  effective  measurement.  Thus,  a primary  re- 
search goal  is  the  development  of  automatic  performance  measurement 
technology.  This  includes  the  development  of  data  on  the  types  of  measures 
needed,  the  use  of  single  and  combined  scores  for  single  and  multiple  task 
dimensions,  weighting  requirements,  etc.  The  basic  questions  are  the 
same:  what  should  be  measured,  how  do  you  measure,  when  do  you  meas- 

ure, and  how  do  you  organize  the  measurement  information?  Underlying 
these  are  the  issues  of  reliability,  validity  and  sensitivity  of  the  measures. 

The  selection  of  performance  standards  (e.g.  , RMS,  mean 
absolute  error,  time-in-tolerance,  etc.)  for  training  system  design  cannot 
be  made  on  any  systematic  basis,  thus  a considerable  research  effort  is 
indicated  here.  At  the  present  time,  error  measurement  is  of  some  con- 
cern based  both  on  the  paucity  of  design  data  and  on  certain  concepts  of 
adaptive  systems  (for  example,  in  Kelley's  technique,  information  on  the 
distribution  of  error  in  trainee  performance  is  not  available). 

d.  The  adaptive  logic  requirements  are  of  course  another  major 
contributor  to  the  success  of  adaptive  sequencing.  A great  deal  of  design 
data  is  required  on  criterion  limits  for  various  measures  and  for  stages 
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TM*  Chapter  pr«?*<p«8*  concepts  a«d  »«?«£ n>  for  deliberately  de- 
parting from  realism  in  design  Sn  order  to  enhance  training  effectiveness • 
TtH>*«>  r*>- for  to  |rf*«ttt#tf  »fevii4Uflns  »n  <««!*«’« r*W««  or  operation  *•»«*;  »at*«l 
m-iflln  8he  opera».io»*al  «im*ilaiod  to  aid  learnin#*  a«»d  iltua  differ 

from  a *o»8 •eft** live  lo»»ri>n«  'if  iH#  e«uine«rion  fideliiy  or  a reduction  in 
fideti;«y  «olerance#»  u^nere  the  enfjiueer&nu  »8ate-of-lf»e-*rt  i*  Im*  than  ade- 
c|oaie.  The  antrUif^ni  u#e  of  thi*  type  of  dealer?  option  serve*  to  increaae 
performance  <t»y  jvrovidln®  condition*  for  learning  to  occur;  it  i*  also  a prime 
mean*  for  moJivatin*  student*  to  perform  since  incentive*  vdiich  motivate 
man"*  activities  come  in  part  from  the  consequence*  of  hi*  own  action*  a* 
they  are  understood.  The  issue*  of  concern  in  this  design  approach  are 
the  following-! 

• Informal l«m  feedhack»-“this  considers  the  reteatts  for 

providing  the  feedback  of  performance  information 
via  display  design  to  the  trainee.  Included  also  are 
provisions  for  extra -mission  equipment  in  the  trainee 
station. 

. Guidance  support*- -thi*  concern*  the  provision  for 
cues  and  prompts  in  training  equipment  to  facilitate 
performance . 

. Display  enhancement** -this  refers  to  techniques  for 
stimulus  coding,  concerned  with  enhancing  identifiable 
coding  dimensions  by  which  information  can  be  displayed 
optimally  to  the  trainee  in  order  to  increase  training 
efficiency.  These  techniques  provide  for  enhancement 
to  insure  effective  signal  monitoring,  detection  and 
identification.  A discussion  of  these  issues  has  already 
been  provided  in  paragraph  3.4 . 3 of  this  report. 
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* •>  *•  * f-tV«  u =5  5^4^  qu*|  ^>r«  'ijtSMS 

e^im* f*m*ti!.*  * ■»  * -*»vf«i  «h«  pat-po**  wf  i.r  aining  and  the 
l#»5*i>f>h!g  I.-mf;  ft  iw#  a gi  tnen  tf»¥t*»*vg  *yAl«m» 

J,  5\  9 rgg4»a^  fc  . A 4*»ft5red  design  goal  5ft  ««  provide  Ihf 

t.Apabilifty  5»  «q«ip#*ient  f«)fTh(?  sMcM  lO  know  («ft  («fli«M!ftl)  how  the  r«-» 
s««|ft.*  «.j  hi*  fN>r(ormiift«.#  iftitofm  lo  «K|w>iMU0ft«  or  w»rm#.  Mw.lt  evidence 
ha*  been  **ftembl*d  wvi?r  many  >p»r»  and  in  many  job  comfit*  ihai  indi- 
cal***  Mwb  ft-s.loMi ry  «*f(ectft  o(  knowledge  o(  rc*«U#  or  information  feedback 
on  learning  and  on  performance . 

Considerable  frftftArc.lt  d(or«  has  been  i>S|wfl«J#d  on  dlUf«r«niiaiin| 
iHc  basic  meaning*  of  the  term*  And  on  d«ttn«a(lflg  lift  empirical  And  theo- 
reiical  properiiei,  And  a nember  of  eKperimenw  have  examined  ihe  fre- 
quency  and  temporal  condition*  relative  to  providing  information  feedback  in 
various  task  situations  (see,  for  example,  the  authoritative  review  by 
Bilodeau,  l%b;  and  report*  by  Anneu  and  Paterson,  196b;  Micheli,  1966; 
Adam*.  1964;  Anneu,  1961;  Smode,  1958;  and  Ammon*,  1956).  Bilodeau 
and  Bilodeau  (1961),  in  their  review  of  motor  skill*  learning,  concluded 
that  information  feedback  i*  the  strongest  and  most  important  variable  in 
controlling  performance  and  learning. 

The  evidence  on  the  efficacy  of  information  feedback  on  perfor- 
mance when  it  is  provided  in  relevant  and  meaningful  ways  is  quite  clear 
(other  than  occasional  contentiousness  over  quite  detailed  relationships 
belabored  »n  some  laboratory  studies)  and  a review  of  the  findings  need 
not  be  undertaken  here.  In  essence,  the  highlights  of  information  feedback 
pertinent  to  design  are  die  following: 

• Learning  becomes  more  efficient  (increases  in  the  rate  and 
in  the  level  of  performance)  as  the  precision  of  relevant 
information  increases.  Several  sources  of  information 
have  been  identified  in  the  regulation  of  performance: 
intrinsic  feedback  which  is  the  information  that  the  trainee 
receives  from  his  own  internal  movements:  action  feed- 
back  which  consists  of  externally  displayed  cues  inherent 
in  a task;  and  augmented  feedback  which  refers  to  the 
availability  of  special  information  not  found  in  the  task 
which  may  be  provided  by  an  instructor  or  by  additional 
feedback  circuits  in  training  equipment. 

• Information  feedback  serves  three  functions:  a directive 
effect  on  performance  (i.e. , student  action  to  correct 
error),  a motivating  effect  and  a reinforcing  effect  on 
performance. 
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• The  student  may  perform  more  nearly  it*  hi#  capacity  when 
provided  wish  i#f«rm*u«m  on  bow  be  ?•#  dc>i®g  «*«»xdi  how  hi#  per* 
forman*  e conform*  to  eape*  tatfon#  » 

• Augmented  feedback  HAd*  to  im  reasingly  »** 

either  (titwrcm  or  incoherent  tasks,  particularly  during 
vanced  stage#  of  learning  bee a use  of  the  mere  precise  iufer- 
nuiion  given  to  the  Ip<*  rw: , 

• H is  indie aied  Uat  augmented  feedback  increase#  the  rate 
of  improvement  early  in  task  prat  tic # and  that  it  rai#e« 
the  level  of  performance  in  overtea rned  tasks.  It  also 
account#  for  the  frequent  reports  to  the  effect  that  task# 
are  more  interesting  and  less  fatiguing  when  achievement 
information  i#  provided  than  when  this  information  is 
withheld.  Generally,  the  more  precise  the  information 
and  the  more  immediate  the  presentation,  the  greater 
the  facilitating  effect  on  performance.  The  Important 
feature  is  the  recognition  that  the  ultimate  performance 
criterion  is  often  too  gross  for  the  trainee  to  appreciate 
increments  of  improvement.  What  is  required  is  moment - 
to ’ moment  availability  of  relevant  information  on  how  the 
results  of  performance  conform  to  some  objective  reference. 


Since  training  device  d«*i|n  ia  at  i«*H®  her®,  ©?»f  interest  centers 

on  the  display  of  augmented  feedback  to  the  trainee.  *r‘hf3  is  a special  case 
of  information  presentation  in  which  the  student  is  provided  a signal  imme- 
diately after  responding  which  indicates  the  adequacy,  correctness,  or 
accuracy  of  his  performance.  Tide  information  is  not  available  in  the 
learning  task  per  se,  but  is  provided  via  control  from  the  instructional 
system. 


With  the  obvious  advantages  for  training  afforded  by  the  informa- 
tion feedback  loop,  the  human  factors  issues  concern  the  possible  means 
for  implementation  and  display  in  training  equipment.  A number  of  design 
options  are  available  for  providing  supplementary  or  augmented  information 
feedback  to  the  trainee. 

3.7. 1. 1 Error  Information  Provided  in  Primary  Displays.  Evidence 
indicates  that  skill  learning  is  more  efficient  with  dynamic  information 
feedback  (immediately  as  accrued)  then  when  this  supplementary  or  extra 
information  is  withheld  or  when  knowledge  of  performance  is  provided  after 
task  completion.  With  the  automatic  computer  scoring  capability  , error 
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a»  Aitiiwd,  Can  Ik?  displayed  ©n  a pri.Mry  5rat«ee  display. 

Tb»s  i»  applicable  (6  all  Ifaittldg  devices  in  which  the  sludvflt(s) 

•utilises  electronic  displays  in  j«b  }Mffdrminc«.  A typical  example  of  this 
v««  it  fojnd  in  the  Simulator  tor  Electronic  Warfare  Training  (SEWf), 
described  earlier  in  thi*  report,  wherein  it  «*  proposed  tlai  errors  com* 
mi  tied  by  the  train*  in  emitter  signal  analysis  he  shown,  m alpha-numeric 
form,  on  the  CRT  display  he  uses  for  emitter  signal  analysis  (specifically 
in  this  cose,  the  AN/ALR«27).  The  information  feedback  is  terse,  provid- 
ing essential  elements  indicating  equipment  locus  and  error  type,  e.g. , 

"A LA -6  WRONG  POLARITY."* 

1.7. 1.1  Displays  of  Information.  Another  form  of  information  augmen- 
tation involves  the  use  of  variations  in  the  number  of  displays  which  provide 
information  for  control  skills.  Laboratory  research  has  found  differences 
in  the  performance  of  simple  tasks  as  a function  of  number  of  displays  em- 
ployed. Tracking  studies  (Briggs,  1961;  Briggs*  1962)  which  varied  the 
scheduling  of  displayed  information  during  the  acquisition  of  vehicle  control 
skills  in  the  sequence  of  training,  found  that  in  the  early  stage  of  learning, 
the  trainee  responded  almost  exclusively  to  system  error  amplitudes  where- 
as with  high  skill  attainment  he  utilised  error  amplitude  and  error  rate 
information  and  also  determined  his  controlling  responses  by  reference  to 
the  acceleration  characteristics  of  system  error.  Briggs  implies  that  the 
trainee  requires  more  displayed  information  on  the  derivatives  of  system 
error  early  in  training  than  he  does  at  a later  stage,  based  on  the  following 
reasoning:  "A  display  in  instantaneous  system  error  amplitude  contains 

information  on  error  rate  and  acceleration  which  the  operator  himself  may 
estimate  by  observing  the  display  indications  over  time  (the  rates  and  accel- 
erations of  cursor  movement).  Thus,  it  is  possible  to  maintain  a reasonably 
accurate  heading  In  an  aircraft  by  reference  Only  te  the  jjy SO  iSHij*SS6 , ®iU€6 
the  pilot  can  observe  not  only  the  extent  (amplitude)  of  heading  orror  at  any 
time,  but  also,  he  can  note  the  rate  at  which  the  compass  is  rotating  and  the 
extent  to  which  his  flight  path  is  accelerating  away  from  the  desired  heading. 
H v.vever,  it  is  obvious  that  only  the  highly  skilled  pilot  can  operate  effec- 
tively under  such  partial-panel  conditions,  and  In  order  to  hold  a particular 
course  without  undue  oscillations  around  the  desired  heading,  the  less 
skilled  operator  requires  the  bank  and  turn  indicator  plus  the  artificial 
horixon  (displayed  derivative  Information)."  Briggs  (1962,  pg.  1).  A 
follow-on  study  examined  the  hypothesis  that  a training  schedule  which  pro- 
gressively reduces  the  number  of  displays  used  in  vehicle  control  (two 
dimensional  tracking  task)  will  enhance  training  compared  with  training 
that  begins  with  a reduced  number  of  displays.  However,  the  display  of 
visual  information  on  error  rate,  error  acceleration  and  information  on 
error  during  training  did  not  enhance  transfer  to  displays  showing  error 
only.  The  lack  of  significance  in  the  results  was  in  part  attributed  to  the 
2-display  task  being  too  easy. 
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3.7.1.  I Auidimicd  Information  FeedbitK  Display,  the  moat  recent 
re»r*rcb  emtthaiiita'K«rit<fih«  computer  control  of  information  fretlbatk 
to  tht*  *tude«t»based  on  an  automated  feedback/ guidance  capability  and  an 
automated  monitor,  evaluation  and  scoring  system.  A study  by  Facontt, 
Mortimer  and  Simpson  (H70),  exemplifies  this  effort.  They  examined, 
in  a flight  simulation  context,  the  feasibility  of  various  automated  approaches 
to  presiding  information  feedback  and  guidance  to  the  student  via  system 
hardware.  The  automated  feedback  and  guidance  capability  is  predicated 
on  an  automated  moni(oring,e valuation  and  scoring  system  in  the  training 
device  which  provides  the  signals  for  defining  and  selecting  the  message  re- 
quirements as  a function  of  student  error  In  relation  to  preset  tolerances. 
Also,  the  automated  measurement  system  provides  a priority  signal  (when 
required)  which  determines  which  message  to  output  in  the  event  more  than 
one  is  pending.  Within  the  audio,  visual  and  proprioceptive  modalities,  the 
available  information  feedback  and  guidance  approaches  are  excetoted  in  the 
following.  A summary  of  these  feedback  and  guidance  approach"  is  also 
provided  in  Table  34. 

Audio 


Composed  Messages  (Words /Syllables).  The  sequence  of 
words/syliables  to  compose  the  required  message  is  stored 
digitally  in  computer  core,  disk,  or  drum.  All  possible 
messages  are  stored  in  memory,  and  the  proficiency 
measurement  programs  will  identify  the  required  message, 
specifying  the  message  number,  code,  or  address.  The 
system  will  decode  the  number  and  thereby  determine  the 
word  sequence  and  output  the  correct  sequence. 


Composed  Messages  (Phonetics).  This  approach  stores 
phonetic  sounds  and  composes  each  word  and  each  mes- 
sage by  sequencing  the  correct  sounds.  Each  message 
stored  contains  the  sequence  of  sounds  and  the  spacing 
to  construct  messages. 


Nonverbal  Messages.  This  approach  uses  an  audio 
frequency  as  the  information  feedback.  The  evaluation 
program,  based  on  trainee  response,  selects  an  audio 
frequency  b~nd  corresponding  to  the  parameter  out-of- 
tolerance.  The  deviation  of  the  trainee's  response  from 
the  desired  response  is  proportional  to  the  change  in  audio 
frequency  from  the  center  frequency  of  the  selected  fre- 
quency band.  For  example,  in  the  flight  simulator,  when 
the  pitch  angle  exceeds  tolerances,  the  performance 
evaluation  program  determines  tint  pitch  angle  is  in 
error  and  selects  the  frequency  band  corresponding  to 
pitch  angle  being  in  error. 
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TABLE  34.  SUMMARY  OF  FEEDBACK  AND  GUIDANCE  APPROACHES 
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(from  Facontij  Mortimer  and  Simpson,  1970.) 
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. Audio  Insetting.  This  approach  assumes  a small  number  of 
standard  formats  for  the  total  message  requirement.  The 
standard  formats  are  stored  on  some  continuous  type  of 
audio  medium  (tape,  cartridge,  etc.).  The  key  words 
describing  the  parameter  of  interest  and  the  direction 
of  deviation  are  inserted  in  the  appropriate  position. 

# Continuous  Messages.  In  this  approach,  each  feedback 
message  is  stored  as  an  addressable  item,  and  for  each 
feedback  message  desired,  a storage  slot  is  required. 

Visual 


# Model.  For  flight  simulation,  this  approach  presents  an 
aircraft  model  to  the  trainee  and  feedback  is  achieved  by 
moving  the  model  along  the  axis  which  is  found  to  be  out- 
of-tolerance  in  a manner  that  indicates  the  corrective 
action  required.  A closed  circuit  television  (CCTV) 
technique  is  used  to  present  the  model  to  the  trainee, 
with  the  image  presented  on  a heads-up  display  to  re- 
duce the  attention  requirements  on  the  trainee.  The 
use  of  a physical  model,  however,  provides  some 
distortion.  For  example,  if  the  model's  longitudinal 
axis  is  aligned  with  the  aircraft  longitudinal  axis,  small 
changes  in  pitch  will  be  undetectable.  Therefore,  if  a 
change  in  pitch  is  required,  the  model  is  rotated  about 
the  yaw  axis  to  present  a side  view  to  the  trainee. 

. Instrument  Drivers.  This  approach  requires  activation 
of  the  instrument  associated  with  the  parameter  requiring 
correction,  for  altitude  the  altimeter,  for  pitch  or  roll 
the  HPT,  etc.  One  technique  to  accomplish  this  is  to 
develop  a function  to  drive  the  indicator  during  feedback 
cycles.  This  is  accomplished  by  driving  the  instrument 
from  the  actual  value  to  the  desired  value  on  a cyclic 
basis . 

9 Two-pointer  Instruments.  Visual  feedback  may  be  given 
by  modifying  the  instruments.  Each  pointer-type  instru- 
ment could  have  redundant  pointers  with  contrasting  colors. 
The  size  is  also  relevant*,  the  feedback  pointer  must  be 
smaller  in  both  length  and  width  such  that  when  r.ot  in  use 
it  is  not  apparent  on  the  instrument  face  and  can  be  hidden 
behind  the  normal  pointer. 
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• Instrument  Indicator  Lights,  This  approach  utilizes 
miniature  indicator  lights  built  in  the  face  of  selected 
instruments*  The  lights  are  located  in  a circular 
pattern.  When  information  feedback  is  required,  the 
lights  are  energized  sequentially,  thus  giving  the  sen- 
sation of  a moving  light.  The  motion  is  in  the  direction 
of  the  required  corrective  action.  An  adequate  number 
of  lights  is  predicted  to  be  five. 

• Head  Up  Display  (HUD)  - Symbolic.  Feedback  may  be 
provided  through  a HUD  type  device.  One  approach  is 
to  provide  the  feedback  in  the  foxm  of  schematic  repre- 
sentations of  the  parameters.  These  symbols  would  be 
similar  to  those  presently  used  in  the  aircraft  instru- 
ments and  in  conventional  HUD.  Figure  46  provides  an 
example  of  this  approach. 

• Head  Up  Display  (HUD)  - Verbal.  This  approach  uses  the 
HUD  to  generate  English  Language  feedback  messages. 
Basically,  the  operation  is  the  same  as  for  audio  feed- 
back. The  messages  appear  on  a combining  glass  placed 
in  the  trainee's  normal  out- of- the -window  field-of-view. 


Froprioceptive 


. Cutaneous  Feedback.  This  approach  utilizes  the  vibro- 
tactile  sense.  It  can  be  implemented  by  placing  vibrators 
in  contact  with  the  trainee's  skin.  Varying  the  duration 
and  amplitude  of  the  vibrations  provides  an  information 
channel  to  the  man.  The  position  on  the  body  can  be  used 
to  indicate  the  parameter,  and  the  variations,  in  ampli- 
tude and  duration,  make  up  the  message  content. 


3.7.  1.4  Supplementary  Signals.  Other  design  possibilities  are  available 
for  augmenting  the  feedback  loop  during  training.  These  include  the  pro- 
vision for  various  signals  to  indicate:  that  performance  error  has  exceeded 

criterion  limits;  that  the  trend  of  performance  is  approaching  an  out-of- 
tolerance  condition;  or  that  a procedural  error  has  occurred.  To  this  end, 
auditory  signals  (alerting  sounds,  prerecorded  words,  computer-generated 
sounds  based  on  voice  synthesis),  visual  signals  peripheral  to  the  trainee's 
attention  (lights),  or  visual  signals  overlaid  on  primary  displays  (discrete, 
patterned)  are  easily  implemented.  This  type  of  supplementary  information 
is  of  particular  value  during  initial  stages  of  learning  by  delimiting  the  fre- 
quency and  severity  of  error  and  limiting  the  exploration  time  in  performance 
One  purpose  of  this  signalling  is  to  alert  the  usually  task-overloaded  trainee 
to  certain  critical  errors  and  error  buildups  that  may  have  a cumulative 
effect  on  performance,  if  not  modified  in  a timely  way.  This  type  of 
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Figure  46.  HUD  Symbolic  Feedback 
(from  Faconti,  Mortimer  and  Simpson,  1970) 
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performance  information  usually  should  not  continue  to  the  point  where  the 
trainee  becomes  dependent  on  extra  information  in  job  performance. 

3.7.  1.5  Extra-Mission  Equipment . Another  means  for  optimizing  train- 
ing opportunity  via  design  is  the  provision  for  extra-mission  equipments  in 
the  trainee  compartment.  These  refer  to  hardware  and  associated  training 
functions  which  are  not  present  in  operator  stations  in  the  actual  system. 
Their  purpose  is  to  increase  training  flexibility  and  to  enhance  learning 
through  the  use  of  supplemental  information  (for  demonstration  and  for  in- 
formation feedback)  not  available  on  the  job.  This  feature,  one  of  a grow- 
ing number  that  emphasizes  the  Mtr aining  tool11  characteristics  of  simulators 
is  becoming  increasingly  prevalent  in  the  design  specifications  for  new  train 
ing  devices  . 


A representative  example  of  this  facet  of  instructional  control 
is  afforded  by  Device  2B24  (helicopter  instrument  flight  trainer,  described 
eariler  in  this  report).  A capability  for  demonstrating  training  problem 
solutions  is  provided  in  the  trainee  cockpit.  Upon  the  request  of  the  trainee 
or  at  the  discretion  of  the  instructor,  demonstrations  can  be  executed  such 
that  cockpit  instruments  and  the  motion  module  exhibit  movements  as  actu- 
ally flown.  This  may  be  in  real  time  or  slow  time  or  combinations  of  both. 
A pre-recorded  verbal  explanation  and  commentary  synchronized  with  the 
flight  is  provided.  The  demonstration  emphasizes  both  didactic  quality  and 
technical  accuracy  to  insure  trainee  understanding.  The  following  problem 
solutions  or  maneuvers  are  envisaged  to  be  programed. 


• The  interpretation  of  altitude  and  flight  control  instruments, 
including  the  relationships  between  displayed  information 
and  control  movements. 

• VOR  approach.,  including  reporting  procedures  and  exe- 
cution of  a missed  approach. 

• A tactical  ADF  approach,  including  reporting  procedures, 
wind  drift  correction,  and  execution  of  a missed  approach. 

* A standard  ADF  approach,  including  reporting  procedures, 
wind  drift  correction,  and  execution  of  a missed  approach. 

* An  ILS  approach,  including  wind  drift  correction,  report- 
ing procedures,  and  execution  of  a missed  approach. 

* Control  of  the  aircraft  during  climbs,  descents,  turns, 
climbing  and  descending  turns  and  level  flight  by  refer- 
ence to  attitude  instruments. 

. Interpretation  of  navigation  instruments. 


- 416  - 


NAVTRADEVCEN  69-C-0298-1 


* Holding  at  an  intersection,  including  entry  from  90° 
off-axis,  wind  drift  correction  and  departure  at  a 
predetermined  time  . 

# An  in-flight  engine  failure  and  air  restart;  an 
instrument  takeoff. 

The  "trainer  peculiar"  equipments  in  the  cockpit  include:  an  information 

display  panel  for  relevant  information  (status  and  feedback  information  pro- 
vided the  trainee);  an  automatic  program  control  panel  for  initiating  certain 
events  (e.g.  , demonstration);  a CRT  monitor  (monitored  by  the  trainee  for 
graphic  data  and  for  debriefing);  and  CCTV  camera  (for  instructor  viewing 
of  engine  and  attitude  instruments  and  trainee  manipulation  of  aircraft 
controls).  Voice  alerts  are  provided  (via  the  intercom)  on  parameters  of 
flight  which  are  outside  acceptable  tolerances  for  a given  maneuver  or  flight 
condition.  In  addition,  pre -mission  briefing  is  automated  and  provided  to 
each  trainee  while  seated  in  the  cockpit  (Hundt,  1969). 

The  proposed  simulator  for  electronic  warfare  training  (SEWT, 
described  earlier  i n this  report)  possesses  a demonstration  linkage  between 
instructor  and  student  to  facilitate  the  development  of  an  optimum  training 
strategy  per  trainee.  Preprogramed  emitter  signatures  as  well  as  specific 
characteristics  of  the  preprogramed  emitters  can  be  demons  tr  a the 

trainee  compartment  either  during  briefing/ pre -mis sion  setup  or  when 
trainee  performance  is  below  expectations  during  any  segment  of  a specific 
mission  in  the  training  sequence. 

Another  form  of  extra-mission  equipment  in  the  trainer  com- 
partment is  the  annunciator  panel  proposed  for  the  Multiple  Trainee  Station 
Air  Navigation  Trainer,  Device  1D23  (Bark,  et  al,  l 969).  This  offers  a 
means  of  precise  stimulus  control  for  training  and  also  provides  immediate 
supplemental  information  to  the  trainee  about  the  results  of  his  performance 
A number  of  programed  questions  are  displayed  to  the  student  requesting 
certain  information  and  specific  types  of  behavior  (e.g.  , calculate  bingo 
fuel  and  time,  calculate  ETA  to  the  next  fly-to  point).  Each  question  is 
programed  to  be  visually  displayed  at  specified  times  in  the  mission  cycle 
and  remains  ,,onM  for  30  seconds  during  which  latency  of  response  is  re- 
corded up  to  2 minutes.  Failure  of  the  student  to  respond  (via  keyboard 
entry)  to  the  question  in  this  time  frame  results  in  negative  knowledge  of 
results.  In  all  cases,  information  feedback  is  given  to  the  student,  via  the 
computer,  immediately  upon  completion  of  response  to  each  annunciator 
panel  question. 

A performance  measurement  technique  that  also  provides  the 
student  supplemental  information  about  performance  not  available  in  the 
job  context  is  the  trainee  monitoring  system  ( TMS)  designed  for  installation 
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in  the*  cockpit  of  operational  flight  trainers.  * Procedures -following  se- 
quence* (computer  monitored  checklist  events)  involving  both  normal  and 
emergency  procedures  in  the  mission  cycle  are  displayed  alpha-numeri- 
eaily  on  a CRT.  The  trainee  is  provided  information  on  the  adequacy  of 
his  procedural  performance  as  accrued  (i.e.,  in-tolerance  performance 
and  errors  of  omission  and  commission  in  procedural  checklist  activities 
displayed  on  the  TMS  CRT).  This  information  is  also  recorded  for  per- 
formance measurement  purposes. 

3.7.2  Guidance  Supports.  Providing  the  trainee  knowledge  about  per- 
formance immediately  as  accrued  is  a desirable  design  option  in  a number 
of  training  situations.  Another  means  of  providing  supplementary  informa- 
tion which  has  similar  utility  for  design  involves  the  use  of  cues  or  prompts, 
which  provides  the  trainee  information  about  performance  before  or  during 
his  responding.  Techniques  of  cueing  or  prompting,  whereby  guidance  is 
provided  via  the  training  equipment,  have  not  been  employed  with  any  effect 
in  training  devices.  However,  the  value  of  this  approach  for  learning  has 
been  demonstrated  clearly  and  the  importance  of  this  means  of  support  is 
well  recognized.  Our  interest  concerns  noth  the  design  of  supports  to  be 
provided  to  the  trainee  and  the  assistance  provided  to  the  instructor  (alerts, 
trend  information)  which  will  simplify  and  improve  his  instructional 
functions . 

Considerable  research  on  learning  has  indicated  that  task  relevant 
guidance  given  to  the  trainee  (i.e.  , stimulus  support)  is  particularly  use- 
ful during  initial  learning  for  it  emphasizes  the  relevant  aspects  of  perfor- 
mance, limits  the  exploration  time  and  minimizes  gross  errors  in  perfor- 
mance. As  learning  progresses,  these  cues/prompts  are  eventually 
withdrawn  until  the  trainee  responds  to  the  stimuli  found  in  terminal  per- 
formance (see  Skinner,  I960). 

Providing  cues  and  prompts  via  equipment  is  apparently  more 
useful  in  part  task  trainers  than  in  complex  weapon  system  trainers  (such 
as  multi-man  tactical  training  devices  which  place  a premium  on  engineer- 
ing fidelity  and  one-to-one  transfer  of  training  from  the  device  to  the  oper- 
ational situation).  In  a study  of  sonar -like  detection  training  (short  bursts 
of  tone  presented  at  irregular  intervals  on  a background  of  white  noise), 
Annett  (cited  in  Annett  and  Paterson,  1966)  found  that  presenting  a visual 
warning  cue  prior  to  each  signal  tone  resulted  in  positive  transfer  to  an 
uncued  standard  task.  This  cueing  was  found  to  be  more  effective  than 
either  immediate  or  summary  knowledge  of  results.  Annett  and  Clarkson 


1MIL.  T-82335 , Military  Specification,  Trainers,  Flight:  General  Specifi- 

cation for.  Also  specified  as  a requirement  in  the  design  for  Device 
2F100,  Operational  Flight  Trainer  (A-7E  Aircraft)  . Specification  311- 
1112,  U.S.  Naval  Training  Device  Center  j Orlando,  Florida;  13  November 
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(1964)  suggest  that  cueing  is  a feasible  technique  for  training  in  perceptual 
and  monitoring  tasks  (e.g.  , sonar  and  radar  watchkeeping)  since  training 
can  be  arranged  to  give  the  student  maximum  exposure  to  the  relevant  sti- 
mulus aspects  of  the  task.  The  experimental  task  required  the  detection 
of  a weak  near -threshold  auditory  signal  (1000  cps  lasting  50  ms).  A visual 
display  consisting  of  four  colored  lights  provided  cue,  correct  response, 
incorrect  response  and  missed  signal  information  (i.e.  , cue  and  knowledge 
of  results  information).  The  superiority  of  cueing  over  knowledge  of  results 
was  demonstrated.  Both  techniques  showed  an  increase  in  detections  but 
also  encouraged  different  patterns  of  response  by  the  trainee.  With  cueing, 
a decrease  in  percent  error  (false  positive  judgments  as  a percent  of  total 
response)  was  obtained,  whereas  with  knowledge  of  results,  an  increase  in 
percent  error  was  obtained  (i.e.  , more  errors  of  commission).  With  the 
caution  that  the  task  was  a simple  monitoring  situation  (lacking  the  variety 
and  intensity  of  the  signal  environment  in  actual  watchkeeping  situations  , 
and  with  shorter  vigil  periods),  the  authors  concluded  that  cueing  provided 
effective  exposure  to  authenticated  signal  samples  for  the  subject  to  reliably 
learn  to  distinguish  between  signals  and  non-signals.  Maintaining  perfor- 
mance by  learning  the  statistical  distribution  of  signals  during  a watch 
period  was  facilitated  by  maximum  exposure  to  signals  (i.e.  , few  or  none 
of  the  signals  were  missed).  It  seems  probable  that  what  was  learned  about 
the  signal  distribution  is  as  important  as  what  was  learned  about  the  signal 
itself. 


Various  studies  concerned  with  signal  detectability  have  indicated 
that  the  use  of  prompts,  cues,  signal  specification,  etc.  , improve  auditory 
detection  performance  (see,  for  example,  studies  by  Gundy,  1961;  Swets , 
1962;  and  Lukaszewski  and  Elliot,  1962).  The  results  indicate  some  value 
in  identifying  and  authenticating  the  signal  prior  to  its  onset,  however,  the 
studies  involved  a number  of  independent  variables  and  the  effects  of  the 
signal  specification  on  performance  were  not  uniformly  clear. 

For  our  purposes,  the  function  of  cueing  and  of  knowledge  of  per- 
formance is  to  provide  information.  Both  techniques  are  desirable  for 
training  design.  The  question  is,  what  guidelines  exist  for  design?  The 
work  of  Annett  and  his  associates  found  systematic  differences  between 
cueing  and  knowledge  of  results  for  both  detection  and  discrimination  tasks 
which  they  interpreted  to  result  from  changes  in  the  response  criterion. 

In  detection  tasks,  knowledge  of  results  increases  detection  by  increasing 
the  response  rate  and  hence  the  false  positive  rate,  whereas  cueing  in- 
creases detection  without  increasing  response  rate  or  the  false  positive 
rate . The  value  of  cueing  is  shown  by  a lowered  threshold  of  the  auditory 
signal,  hence  the  trainee  gets  maximum  experience  with  the  signal,  and  is 
able  to  build  up  a knowledge  of  the  signal  characteristics  more  effectively 
than  by  means  of  knowledge  of  results.  Cueing  also  provides  a good  indi- 
cation of  signal  distribution  parameters. 
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In  essence,  tlie  feature  for  design  is  that  providing  subsidiary  in- 
formation useful  to,  or  about  performance>enhances  learning*  Cues/ 
prompts  ("prior  to")  and  information  feedback  ( "during  or  immediately 
after"  performance)  are  useful  techniques*  It  may  be  that  a mixing  of  cue- 
ing and  information  feedback  is  optimal  for  training  (i.e.  , cueing  followed 
by  augmented  information  feedback).  However,  Annett  and  Paterson  (1967) 
found  that  cueing,  knowledge  of  results  and  a combination  of  the  two  re- 
sulted in  essentially  equal  but  improved  performance  over  a control  group 
for  pitch  and  intensity  discrimination  in  a sonar-like  watchkeeping  task. 
There  is  not  enough  evidence  to  specify  in  detail  when  to  employ  cueing 
and  prompts  and  in  what  forms,  in  training  device  design.  Nevertheless, 
the  reasonableness  of  the  cueing  feature  is  not  easily  discounted  particu- 
larly when  the  instructor  is  afforded  control  over  its  usage  (for  example, 
employed  during  initial  training  and  faded  out  as  appropriate  as  skill  is 
achieved)  . 

A number  of  techniques  have  been  employed  in  training  that  can 
be  assembled  under  the  rubric  of  guidance  support.  The  use  of  the  demon- 
stration mode  in  training  (described  previously  in  paragraph  3.7. 1.5)  is 
clearly  a technique  of  instructional  guidance.  Another  design  technique 
involves  the  use  of  information  displays  which  present  future  or  predicted 
information  based  on  trainee  control  performance  of  the  moment.  The 
predictor  instrument  which  provides  a display  of  the  response  character- 
istic of  the  system  in  terms  of  predicted  system  output  at  some  time  in  the 
immediate  future  is  a representative  example.  The  control  task  can  be 
simplified  by  providing  the  operator  information  relative  to  the  future  of  the 
variable  being  controlled.  This  support  is  of  particular  value  during  initial 
learning  (see  Kelley,  I960). 

Related  to  this  is  a type  of  information  display  which  provides  an 
indication  to  the  trainee  of  the  immediate  or  successive  actions  required 
to  maintain  an  in- tolerance  condition.  This  approach  can  be  implemented 
in  various  ways,  for  example,  by  means  of  miniature  indicator  lights  im- 
bedded in  an  instrument  display  which  provide  guidance  information  via  a 
sequencing  of  the  lights  in  the  direction  of  the  correction  required.  The 
rate  of  sequencing  is  proportional  to  the  magnitude  of  the  error.  A speci- 
fic application  of  this  is  a proposal  by  Faconti,  Mortimer  and  Simpson 
(1970)  (cited  earlier)  for  a circular  pattern  of  miniature  lights  built  into 
the  face  of  an  aircraft  instrument.  The  lights  can  be  energized  sequen- 
tially to  give  the  sensation  of  a moving  light  in  the  direction  of  a required 
corrective  action.  The  lights  are  not  visible  when  performance  is  in- 
tolerance . 

Guidance  may  also  be  built  directly  into  the  controls  operated  by 
the  student.  Controls  can  be  mechanized  to  describe  a desired  action,  for 
example,  a pilot  trainee  "following  through"  on  an  automated  control  stick 
sequence  which  demonstrates  an  ideal  maneuver.  Limits  of  movement 
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can  also  be  built  into  a control  to  insure  that  a trainee  does  not  exceed 
certain  error  boundaries  (for  example , the  extent  of  control  excursions 
is  contained  in  order  to  delimit  the  making  of  gross  errors  in  control). 


3.7.3  Mediation.  A valid  design  alternative  concerns  the  selection  of 
non-equipment  options  as  part  of  the  device  configuration  for  achieving  the 
purpose  and  the  objectives  of  training.  It  should  be  made  clear  that  an 
equally  valid  design  decision  is  to  not  provide  hardware  but  to  use  the  in- 
structor to  perform  certain  training  functions.  We  are  using  the  term 
"mediation"  to  denote  this  design  approach  which  reduces  or  simplifies 
the  device  hardware  and,  in  lieu  of  hardware,  employs  the  instructor  as 
an  integral  part  of  the  training  loop. 


Two  aspects  are  subsumed  under  this  concept:  reduction  (exclu- 

sion) of  hardware  solutions  (use  of  the  instructor)  and  equipment  simplifi- 
cation (e.g.  , a switch  or  potentiometer  control  in  lieu  of  a ship’s  helm, 
paper  and  pencil  representation  for  an  equipment  component,  etc.).  The 
skillful  use  of  these  surrogates  for  actual  equipments  and  the  attenuations 
in  fidelity  of  representation  can: 

• reduce  significantly  the  cost  of  the  training  device 
without  compromising  the  training  effectiveness, 


. pinpoint  the  essential  training  required;  for  example, 
in  a team  training  context,  design  emphasis  is  focused 
on  key  positions  while  supportive  positions  are  provided 
with  only  what  is  needed  to  interface  with  the  primary 
positions  , 

. improve  training  potential  by  structuring  the  training 
situation  and  sharpening  the  focus  on  the  purpose  and 
objectives  of  training. 


3.7.3.  1 Trainee  Station  Options.  A decision  for  trainee  compartment 
design  concerns  the  advantages  in  attenuating  the  fidelity  of  task  repre- 
sentation, both  in  situations  where  a trainee  position  may  be  secondary 
or  supportive  in  a team  context,  or  where  certain  tasks  performed  by  a 
trainee  are  secondary  in  the  pattern  of  training  objectives.  A prominent 
case  can  be  found  in  multi-man  team  training  devices  where  the  design 
alternatives  are  based  on  a selective  training  emphasis  (focusing. training 
on  key  positions  in  a team  while  other  positions  are  only  supportive)  vs. 
an  equal  training  emphasis  (all  positions  are  of  similar  training  impor- 
tance).  A good  example  of  this  departure  from  realism  in  design  is  found 
in  Device  14A2  (ASW  tactical  team  trainer,  described  in  Section  IV  of  this 
report)  . The  purpose  of  this  training  requires  a selective  emphasis  in 
instruction,  which  focuses  on  the  duty  stations  in  three  key  ASW  compart- 
ments (CIC,  UB/plot , and  sonar)  since  the  ASW  operations  are  critical 
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here  and  the  personnel  interactions  are  of  high  density.  The  training  re- 
quirements in  the  other  compartments  (Launcher  Captain  Control  station 
and  the  Bridge)  are  less  demanding  and  hence  are  only  supportive  in  that 
these  trainees  serve  as  necessary  links  in  the  development  of  coordinated 
team  sequences.  Thus,  there  is  no  requirement  for  high  fidelity  in  ship 
steering  dynamics.  The  design  decision  resulted  in  a simple  potentiometer 
control  for  steering  the  ship,  instead  of  the  controls  normally  found  on  a 
ship's  bridge. 

Another  example  of  mediation  in  design  is  found  in  Device  14A6, 
ASW  Coordinated  Tactics  Trainer.  Sonar  contact  reports  are  provided 
through  paper  and  pencil  messages  in  lieu  of  detections  made  from  actual 
sonar  displays.  Theoretically,  this  is  a valid  design  option  since  the  simu- 
lator is  a tactical  decision-making  trainer  involving  large  numbers  of  naval 
units  and  various  force  compositions.  The  sensor  stations  are  relegated 
to  a supportive  role  in  the  overall  training  purpose  and  objectives  and 
sufficient  operational  validity  is  provided  to  achieve  the  training  objectives 
without  synthesizing  the  entire  sonar  facility.  This,  however,  requires 
well  developed  and  precisely  executed  utilization  procedures  (series  of 
training  exercises)  for  the  device. 


3. 7. 3. 2 Instructor  Involvement.  The  use  of  mediation,  involving  the 
instructor  in  lieu  of  training  hardware,  depends  largely  on  the  trainee- 
instructor  relationship  requirements  and  on  tradeoffs  in  equipment  instru- 
mentation. The  decision  on  whether  to  mediate  should  begin  with  the  under- 
standing of  the  trainee -instructor  relationships.  A useful  technique  for 
systematically  examining  where  instructor  mediation  could  be  reasonably 
substituted  for  hardware  is  to  array,  in  matrix  form,  the  mission  or  job 
sequences  against  required  trainee  behaviors.  Cell  entries  in  such  a 
matrix  can  be  made  to  indicate  those  event/ behavior  points  at  which  instruc- 
tor involvement  can  be  effectively  traded  off  for  hardware.  For  example, 
the  variety  of  information  feedback  requirements  to  the  trainee  may  be 
prohibitive  in  terms  of  equipment.  Answers  to  such  questions  as,  "How 
fine  should  the  information  about  performance  be?"  and  "How  complex  should 
the  provided  cues/prompts  be  as  a function  of  level  of  training?"  will 
determine  the  cost/ benefit  of  mediation  solutions  . The  more  complex  the 
behaviors  (relative  richness)  the  more  (logically)  the  instructor  should  be 
involved  in  the  training.  This  tradeoff  depends  on  the  ease  with  which  in- 
strumentation can  be  achieved.  For  example,  if  . the  number  of  categories 
of  information  to  be  provided  the  trainee  during  the  course  of  instruction 
presents  problems  in  storing  the  information,  then  the  instructor  may 
usefully  serve  to  provide  information  to  the  trainee  (as  required)  in  lieu 
of  equipment  presentation.  A case  in  point  is  the  presentation  of  ground- 
to-air  communications  to  pilot  trainees.  To  instrument  feedback  mechan- 
isms for  a reasonable  universe  of  messages  that  might  be  required  ip  the 
total  set  of  training  exercises  incurs  an  inordinate  equipment  burden  - 
whereas  the  use  of.  the  instructor  for  this  purpose  is  at  present,  a more 
logical  solution.  ■ 
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3.7.4  Research  Issues.  A systematic  effort  is  suggested  to  place  in 
perspective  the  transfer  of  training  benefits  that  accrue  from  the  selection 
of  training  device  design  options  that  are  deliberate  departures  from  oper- 
ational realism.  The  positive  values  for  learning  and  performance  resulting 
from  supplementary  and  augmented  information  and  from  signal  enhancement 
techniques  are  well  documented.  The  basic  question  concerns  their  various 
utilities  for  simulator  training  and  how  they  can  be  most  efficiently  imple- 
mented. A number  of  the  techniques  currently  employed  have  been  selected 
more  on  a logical  than  experimental  grounds  or  based  on  results  obtained 
in  task  contexts  quite  dissimilar  to  those  involved  in  complex  synthetic 
training  situations.  A number  of  issues  worthy  of  consideration  are  out- 
lined below. 

a.  A base  of  experimental  data  should  be  accumulated  on  the 
effects  of  supplementary  information  feedback  and  guidance  supports  on 
performance  in  simulator  training.  This  should  center  on  the  kinds  of 

job  situations  and  training  equipments  that  would  benefit  from  this  approach, 
quantitative  data  on  transfer  of  training  effects  should  be  collected. 

b.  The  indications  are  that  guidance  supports  are  of  greater 
training  value  in  part- task  devices  than  in  weapon  system  trainers.  The 
effects  of  guidance  supports  in  generalized/ universal  training  devices 
should  be  examined  vis-a-vis  transfer  of  training  to  the  operational 
counterparts . 

C.  Greater  definition  of  cues,  prompts,  and  signal  specification 
is  required  as  guidelines  for  design.  This  includes  the  varieties  and  forms 
of  relevant  guidance  supports  and  patterns  of  usage  for  design.  A legiti- 
mate question,  for  example,  concerns  the  fineness  of  gradations  of  these 
supports  as  well  as  the  differential  utility  of  this  fineness  for  training,  if 
any . 

d.  A research  issue  of  some  priority  concerns  the  role  of  feed- 
back and  guidance  as  supplementary  information  in  automated  training 
situations.  The  questions  concern  the  available  options  for  design,  the 
means  of  implementation,  the  credibility  of  the  information  (realistic, 
timely,  sufficient)  that  is  automatically  presented,  and  the  training  values 
afforded  by  this  approach. 

e.  A straightforward  requirement  is  to  put  into  perspective  the 
range  of  information  feedback,  guidance  support,  and  signal  enhancement 
techniques  available  within  the  state-of-the-art,  the  means  for  their  im- 
plementation and  the  relative  cost  considerations  . 
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SECTION  IV 


DEMONSTRATION  OF  HUMAN  FACTORS 
RECOMMENDATIONS  IN  THE  DESIGN  OF  A COMPLEX 
SHORE -BASED  TEAM  TRAINER 

4.  INTRODUCTION 

The  purpose  of  this  section  is  to  provide  a well  developed  example  of 
human  factors  involvement  in  the  training  system  design  process.  A 
demonstration  is  provided  of  the  human  factors  inputs  to  the  design  of 
device  14A2 , ASROC/ASW  Early  Attack  Weapon  System  Trainer.  What 
is  presented  is  a reconstruction  of  the  human  factors  design  process  for 
achieving  the  functional  specifications  for  the  device.  The  points  where 
human  factors  inputs  influence  the  design  of  the  system  are  systematically 
explored,  using  the  method  described  in  Section  II  of  this  report.  Within 
this  framework  of  a human  factors  design  method  overlaid  on  an  operational 
synthetic  training  system,  the  congeries  of  design  issues  are  organized  and 
the  options  available  to  design  are  examined.  The  net  effect  is  an  examina- 
tion of  the  human  factors  design  of  a complex  training  system  already  in 
existence . 

The  luxury  of  viewing  an  on-line  training  system  in  retrospect  provides 
an  opportunity  to  examine  the  credibility  of  the  method  proposed  in  this 
report,  particularly  in  relation  to  the  design  achieved.  It  also  provides 
the  opportunity  to  reconstruct  the  key  design  decision  points  and  to  explore 
the  design  alternatives  to  determine  what  effects,  if  any,  these  modifications 
would  have  had  on  the  configuration  of  the  device  and  on  training  capability. 

The  reconstruction  which  follows  is  based  for  the  most  part  on  independ- 
ent analyses  made  by  Dunlap  and  Associates,  Inc.  , organized  within  the 
method  described  in  Section  II.  Where  use  is  made  of  Navy  reports  or 
other  published  data,  this  information  is  documented. 

The  presentation  is  deliberately  terse,  in  which  the  reader  is  taken 
through  the  steps  in  the  design  process.  Only  the  key  design  decision 
points,  those  that  significantly  influence  the  configuration  and  hence  the  uti- 
lization of  the  device,  are  emphasised,  since  the  definition  of,  and  the 
rationale  for,  the  sequence  is  discussed,  in  Section  II.  The  identification 
and  resolution  of  design  issues  will  not  receive  the  full  blown  treatment  as 
would  be  expected  in  the  human  factors  report  for  any  system  under  study . 
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Thus  , the  highlights  of  the  results  which  are  most  meaningful  to  human 
factors  design  are  stressed.  While  the  highlights  are  outlined  in  a se- 
quential, or  step  series,  it  is  worth  repeating  that  the  process  is  iterative 
in  that  the  data  gathering  and  organization  proceeds  from  gross  to  detailed 
levels  of  analysis  . 

4.1  DEFINITION  OF  PURPOSE  OF  TRAINING.  The  purpose  of  training 
is  defined  by  an  NTDC  team  and  personnel  from  the  cognizant  operational 
using  agency.  The  NTDC  team,  which  prepares  the  Military  Characteristics 
(MC)  for  the  device,  is  usually  made  up  of  the  Project  Officer  (Require- 
ments), an  Engineer  and  a Human  Factors  Specialist. 

The  decision  has  been  made  that  an  ASW  Team  Trainer  is  needed 
and  Fleet  requirements  specify  a shore-based  ASROC/ASW  Early  Attack 
Weapon  System  Trainer  capable  of  indoctrinating  and  training  ASW  surface 
ship  crews  in  ASW  tactics.  From  this  start  point,  a number  of  decisions 
are  made  which  define  clearly  the  type  of  training  system  needed  and  which 
outline  a design  pathway  to  follow  to  most  expeditiously  achieve  a device 
design  consonant  with  the  purpose  of  the  training. 

4.1.1  Characteristics  of  the  Environment  for  Training  and  the  Trainee 
Involvement. 

4.  1.  1.  1 Type  of  Training  Device.  A team  training  device  is  required 
to  provide  training  of  individuals  in  the  team  context  (subteam  and  overall 
team  coordination)  for  destroyer  (DLG,  DDG,  Fram  I)  and  Cruiser  ASW 
operations  . 


Training  emphasis  is  on  procedural  and  tactical  decision- 
making sequences  that  involve  the  coordinated  efforts  of  the  team  in 
operating  and  employing  the  vehicle.  Excluded  is  the  basic  training  of 
individuals  in  their  particular  job  requirements  so  that  the  most  efficient 
use  of  the  14A2  for  training  takes  into  account  that  individual  team  members 
are  already  proficient  in  their  billets.  The  concept  of  team  training  is 
defined  in  this  way:  individuals  perform  in  a structured  job  situation  where- 

in each  member  has  a defined  function  to  perform  interdependent  with  those 
of  the  other  team  members;  the  effects  arising  from  this  interdependence 
of  behaviors  influences  system  performance  . 

Training  will  cover  two  types  of  situations:  1)  where  the  ASW 

events  are  repetitive  and  predictable  and  there  are  specific  and  detailed 
rules  and  procedures  for  handling  them;  2)  where  the  mission  events  are 
unpredictable  and  where  all  action  relevant  to  the  environmental  conditions 


ERIC 


-42  5 


NAV TRADE VCEN  69-C-0298-1 


and  the  state  of  the  system  have  not  been  specified  so  that  operations  do 
not  necessarily  conform  to  standard  procedures.  Both  of  these  situations 
operate  in  a complementary  fashion  in  mission  training. 

What  is  stressed  is  the  development  of  coordinative  skills 
based  on  adequate  individual  skills.  Two  features  of  team  performance 
are  emphasized.  The  first  is  that  team  coordination  develops  and  improves 
when  individually  skillful  trainees  work  in  an  organized  system  environment. 
The  second  is  that  teams  may  be  equivalent  in  overall  performance,  yet 
differ  considerably  in  how  they  go  about  their  business.  Similarly,  in  com- 
plex tactical  situations,  several  equally  good  solutions  to  problems  often 
exist  so  that  differences  between  teams  may  become  evident  only  in  the  way 
a team  copes  with  the  ASW  situation.  Device  design  must  provide  a flexi- 
bility for  installing  these  kinds  of  team  situations. 

Design  Options 

a.  Team  training  would  emphasize  the  procedures  in  target 
detection  and  tracking  with  fire  control  solutions  and 
launch  for  ASROC  and  AWTT  Weapon  Systems,  and  some 
DASH  and  Vectored  attacks  with  aircraft  included.  Train- 
ing sequences  would  be  conducted  in  one-on-one  (own-ship 
and  submarine)  tactical  environments  and  progressing  to 
rudimentary  screening  mission  contexts  and  then  to  search 
and  attack  unit  (SAU)  missions. 

b.  Team  training  would  emphasize  tactical  employment  of  the 
vehicle  over  a substantial  range  of  ASW  missions  involving 
threat  evaluation  and  multi-unit  coordinated  search  and 
attack  operations  as  prescribed  in  Doctrinal  Publications. 
This  embraces  tactical  situations  where  the  events  are 
unpredictable  and  where  all  action  relevant  to  the  engage- 
ment geometry  have  not  been  specified. 

Decision 

Device  design  will  provide  coordinated  training  in  team  pro- 
cedures and  in  tactical  decision  making  such  as  found  in 
operating  and  tactically  employing  an  ASW  vehicle  in  repre- 
sentative ,,hot  war’1  situations.  Team  coordination  will  be 
exercised  to  include  complex  multi-unit  operations  in  SAU 
contexts  involving  a range  of  search  and  attack  plan  operations 
as  presented  in  Allied  Naval  Maneuvering  Instructions  , 

ATP- 1 (A)  . 1 
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Note:  The  Navy  specifications  for  the  Device^  call  for: 

# Basic  team  training  covering  the  operation  of  the  individual 

subsystems,  the  communication  and  coordination  requirements 
in  tactical  situations  . 

m Advanced  team  training  covering  tactical  procedures  in  both 

normal  and  casualty  modes  and  tactical  problems  under  stress- 
ful operational  conditions. 

4.  1.  1.2  Selection  of  ASW  Training  Areas  for  the  Device.  The  trainer 

shall  represent  the  ASW  configuration  and  operational  compartments  and 
equipments  as  defined  for  surface  ships  and  their  ASW  attack  weapons. 

The  compartment  functions  include  target  search  and  detection,  target 
tracking,  threat  evaluation  and  action  selection  and  implementation  as  repre- 
sented by  fire  control  solution  and  weapon  launch.  The  team  training  to  be 
achieved  requires  the  following  shipboard  areas  to  be  simulated  for  training. 

Underwater  Battery  [Plot  (UB/Plot)  — This  is  the  ASW  attack 
center  and  contains  the  attack  consoles  of  the  fire  control  equipment  and 
associated  communications  equipment. 

Sonar- -This  contains  the  equipment  for  sonar  target  acquisition 
and  tracking  and  associated  communications  equipment. 

Combat  Information  Center  (CIC)--This  is  the  center  for  the 
collection,  evaluation,  display  and  dissemination  of  the  overall  tactical 
situation.  Plots  are  maintained  on  friendly  and  hostile  units,  maneuvers 
and  tactics  recommended  and  control  of  the  DASH  vehicle  and  Vectored 
attacks  are  the  responsibilities  of  this  compartment. 

Launcher  Captain fs  Control  Station  (LCCS)  — This  compartment 
ensures  proper  system  attack  progression.  From  here,  the  directions  are 
given  for  positioning  the  launcher,  selecting  a cell  and  selecting  a missile 
or  torpedo  (ASROC,  A WTT)  . Also,  auxiliary  firing  is  accomplished  here 
and  missile  firing  safety  features  are  controlled. 

Conning  Station  (Bridge)- - This  compartment  recommends  and 
approves  orders  for  coordinated  action  of  own -ship  and  conducts  the  man- 
euvering of  own-ship. 

^Specification  for  Trainer,  Weapon  System,  ASROC/ASW  Early  Attack 
Device  14A2  (U),  U.S.  Naval  Training  Device  Center,  323  1-220  , March 
1964.  (Conf.) 
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4.  1 . 1 . 3 Definition  of  System  Components 


Design  Option:  To  provide  specific  system  suites  vs.  general- 

ized training  equipment  for  ASW  operations  . 

Decision:  A direct  transfer  of  training  is  desired  from  Device 

14A2  to  the  ASW  platform  of  the  destroyer  or  cruiser  since  the  teams  eligible 
for  14A2  training  will  be  drawn  from  operational  units  when  these  units  are 
in  port.  The  training  device  will  be  utilized  in  direct  support  of  operational 
destroyer / cruiser  ASW  personnel.  Ideally,  team  practice  in  the  device 
should  be  highly  realistic  to  actual  operations  aboard  ship  and  the  system 
suites  in  the  device  should  be  equivalent  to  those  found  aboard  ship  and  in 
similar  configurations.  Therefore,  the  compartment  spaces,  equipments 
and  operating  modes  in  the  14A2  should  be  similar  to,  or  replicas  of  the 
equipment  suites  aboard  destroyers  (DLG,  DDG,  FRAM  I)  and  Cruisers. 

. SQS-23  Sonar,  Mk  114  Eire  Control  system 

. SQS-23  Sonar,  Mk  111  Fire  Control  system  (as  required) 

(with  provisions  for  the  SQS-^26  to  be  employed  in  the  future) 

• Relevant  equipments  in  the  ASW  compartments  . 

4.  1 . 1 .4  Levels  of  Training.  Emphasis  is  placed  on  the  maintenance  of 

proficiency  of  destroyer/ cruiser  ASW  personnel  in  team  operations  (procedures) 
and  on  advanced  team  training  in  multi-unit  exercises  such  as  found  in  screen- 
ing operations  and  in  SAU  contexts  involving  a representative  range  of  search 
and  attack  operations  . 

4 . 1 . 1 . 5 Training  Emphasis 

Design  Option:  Provide  design  for  an  equality  of  training 

emphasis  among  all  trainees  in  the  team  vs.  design  for  a selective  emphasis 
which  focuses  training  on  key  positions  in  the  team  context. 

Decision:  Training  focus  and  fidelity  of  representation  is  cen- 

tered on  the  duty  stations  in  three  key  ASW  compartments  since  the  critical 
ASW  operations  are  performed  here  and  because  of  the  high  density  of  inter- 
actions among  personnel  in  these  compartments.  These  are:  CIC  , UB/Flot, 

and  Sonar.  The  training  requirements  and  synthetic  training  equipment  needs 
in  the  XjCCS  and  the  Bridge  (Conn)  compartments  are  less  demanding,  either 
in  terms  of  the  frequency  of  involvement  or  of  training  purpose,  for  example, 
no  requirement  for  high  fidelity  in  ship  steering  dynamics.  These  trainees 
serve  most  effectively  as  necessary  links  in  the  development  of  coordinated 
team  sequences  in  the  device. 

The  core  of  the  trainee  population  will  be  individual  ship  per- 
sonnel who  will  be  assigned  and  trained  as  a team  in  the  device  . For  the 
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key  Sonar,  UB/Plot  and  CIC  positions,  personnel  will  already  be  qualified 
in  their  billets.  Other  individuals  may  or  may  not  be  qualified. 


It  is  anticipated  that  the  training  of  individual  ship’s  teams 
will  be  conducted  on  a recurring  basis,  in  that  a ship  will  be  scheduled  for 
the  trainer,  when  in  port,  on  a cyclic  basis  and  as  appropriate  to  fleet 
scheduling.  A team,  for  example,  may  be  assigned  for  two  days  of  training 
on  a quarterly  basis.  Thus,  the  training  exercises  and  the  development  of 
utilization  practices  for  the  device  must  account  for  this  scheduling  in  the 
sequence  of  training. 


4.  1 . 1 . 6 Implications  for  Fidelity  of  Simulation.  There  are  considerable 

requirements  for  procedural  and  for  control  sequences  involved  in  the  funda- 
mentals of  ASW  operations  in  subteam  and  team  performances  . They  center 
on  the  development  of  coordinated  procedures  and  internal  communications 
in  achieving  fire  control  solutions  and  launch  of  the  weapons  systems  avail- 
able. These  requirements  are  also  involved  in  operating  own- ship  when 
exposed  to  basic  submarine  tactics,  torpedo  hydrophone  effects,  equips  ent 
casualty,  and  maneuvering  for  attack  and  lost  contact  procedures  in  a one 
submar  ine  - one  ship  situation.  In  screening  and  SAU  contexts  similar  re- 
quirements exist  under  more  difficult  conditions  and  in  addition,  include 
Vectac  operations,  procedures  in  coordinated  search  and  attack  plans,  and 
inter-unit  communication  procedures.  Thus,  device  configuration  must 
provide  a training  environment  conducive  to  the  realistic  accomplishment 
of  the  following: 

m Procedures  in  search  and  detection 

# Maintenance  of  contact  and  tracking 

# Procedures  for  readying  weapons  systems  for  attack 

# Appropriate  utilization  of  communication  circuits 

. Procedures  in  the  prosecution  of  targets. 


These  operations  require  a high  fidelity  in  the  representation 
of  the  controls  and  displays  of  individual  equipment  components  in  the  sub- 
team spaces.  Replicas  of  much  of  the  operational  equipments  are  required. 
These  are  described  in  Section  4.4,  Gross  Device  Definition. 


Higher  order  tactics  training  involving  tactical  doctrine  and 
team  decision-making  sequences  in  threat  evaluation  and  weapon  assignment 
is,  in  large  part,  mediated  in  the  organization  of  training.  The  development 
of  device  utilization  procedures  will  account  for  structuring  and  controlling 
the  series  of  training  exercises  to  accomplish  the  training  objectives  set 
forth.  As  such,  few  additional  demands  are  placed  on  hardware  to  accom- 
plish this  level  of  training.  There  is,  however,  an  exception  to  this  that 
requires  a major  design  decision.  The  design  option  deals  with  the  issue 
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of  extent  of  simulation  (e.  g.  , what/ how  much  to  simulate)  and  is  concerned 
with  the  representation  of  those  simulation  elements  that  provide  the  sonar 
classification  capability.  (This  is  discussed  in  Section  4.5.  1.2.2.) 

4.  1.  1.7  Device  Dynamics  . Computer  programing  is  required  to 

provide  the  following: 


Own-Ship  Characteristics 


# Simulated  motions  realistically  displayed  on 
sonar , radar,  NC-2  plotter 

# Instrument  readouts  to  correspond  to  ship 
maneuvers -realistic  control  lags  with 
ordered  speed  and  course  changes. 

Other  Vehicle  Characteristics 


Realistic  sensor  displays  of  motion  for 
target  vehicles , surface  ships,  aircraft 


and  DASH 


(speeds,  turn,  radius,  rate  of  change  in 
depth/ altitude  , acceleration/ deceleration)  . 


Weapon  Characteristics 


# Current  weapons  and  their  characteristics 

# Current  weapon  delivery  systems  and  their 
characteristics 

(ASROC,  AWTT.  DASH,  Aircraft) 

# Realistic  weapon  deployment  procedures 

# Post-launch  dynamics  — realistic  display 
representation  (ASROC  trajectory,  weapon 
water  running  times,  torpedo  hydrophone 
effects)  . 

Platform  Motion  Simulation  (Yaw,  pitch,  roll) 

# No  requirement  for  platform  motion  for 
team  training  in  ASW  procedures  and  tactics. 
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Real- World  Visual  Simulation 


, No  requirement  for  outside- the -platform 

visual  representation.  Visual  requirements 
center  on  the  electronic  displays  (Sonar, 

Fire  Control,  Radar). 

4.1.2  Provisions  for  the  Management  of  Training  at  the 

Instructor  Station 

4.  1.2.  1 Instructional  Staff.  Design  should  account  for  three  or  more 

instructors  in  training  exercises:  a chief  instructor  at  the  master  console 

and  two  remote  instructors  located  in  the  UB  plot-sonar  and  CIC  . Additional 
instructors  may  be  employed  at  the  master  console  or  as  needed  in  the 
trainer  compartments  depending  on  the  stage  of  training  and  level  of  exercise 
difficulty. 

4 . 1.2.2  Training  Device  Mode  of  Operation 

Design  Option:  manual  vs,  preprogramed  exercise  sequencing. 

Decision:  selection  of  a manual  mode  for  enroute  exercise 

control.  The  use  of  automated  training  sequences  (preprogramed  scenarios) 
ahd  selected  adaptive  training  exercises  were  considered  and  rejected  in 
order  to  achieve  greater  flexibility  in  instructional  control  to  account  for 
significant  differences  among  teams  in  entry-level  skills  and  for  differences 
in  time  available  for  training  among  teams . 

4.  1.2.3  Pre-Mis*sion  Requirements  . Provision  should  be  made  for  a 

team  briefing/cr itique  space.  Mission  briefing  will  be  conducted  prior  to 
the  actual  conduct  of  each  training  exercise.  Briefings  for  training  exer- 
cises of  a similar  type  (repetitive)  may  be  accomplished  remotely  in  the 
spaces  with  trainees  emplaced.  Communications  capability  for  this  will  be 
provided  (see  Section  4.  5 . 1 . 1)  . 

4 . 1 . 2 . 4 Approach  to  .Monitor  and  Control  of  Training  During  an  Exercise  . 

Manual  control  of  each  exercise  will  be  provided  throughout  the  exercise  (via 
a training  utilization  guide  which  provides  training  exercise  scenarios  grad- 
uated in  difficulty).  The  instructor  is  provided  control,  display  and  commu- 
nications capabilities  to  establish  the  initial  conditions  for  each  exercise, 
to  control  own  ship,  target  and  media  variables  during  the  course  of  the 
exercise  and  to  monitor  trainee  performance.  A projection  display  system 
(showing  a history  of  the  exercise}  is  envisaged  to  provide  instructional 
assists. 
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Performance  Monitoring/ Assessment 


Design  Option:  manual  vs.  automatic  scoring  system. 

Decision:  selection  of  a manual  technique  involving  scoring 

based  on  instructor  judgments  for  assessing  individuals  within  the  team 
and  team  performance.  The  situational  display  of  the  total  mission  via 
the  projection  display  will  provide  indicants  of  system  performance  (e.g.  , 
geometry  of  the  engagement  as  it  unfolds,  and  weapon  firing  in  terms  of 
water  entry  point). 

Design  Option:  record  keeping  of  individual  team  perform- 

ances vs.  only  the  on-site  critique  of  team  performance  after  each  exercise. 

Decision:  Verbal  on-site  critique  using  the  projection  display 

of  the  engagement;  no  quality  control  provisions  involving  additional  device 
hardware  or  computer  software  programing.  Hard  copy  records  of  perform- 
ance (for  critique  purposes  and  school  record  keeping  purposes)  will  not  be 
provided. 

4.  1.2.6  Host-Mission  Requirements.  Critique  of  performance  will  be 

conducted  after  completion  of  each  exercise  for  the  assembled  team  in  the 
briefing/critique  space.  Projection  display  equipment  will  be  employed 
for  mission  reconstruction  of  each  exercise. 

4.  1.2.7  Configuration  of  the  Instructor  Station.  The  problem  control 

room  will  house  the  instructor  station  and  will  provide  space  for  team 
briefing  and  critique  functions.  The  projection  display  system  will  also 
be  located  here. 

The  consoles  which  make  up  the  instructor  station  will  be 
organized  on  the  basis  of  a modular  design  concept  with  indicators  and 
controls  grouped  by  related  functions  on  each  console.  The  following  inte- 
gral modules  are  required:  vehicle  control  (own-ship,  support  ships  and 

aircraft,  target  submarine);  fire  control  (fire  control  system,  launcher 
system  and  weapons);  sonar;  radar;  and  support  (environmental  controls, 
communications,  projection  display  controls). 

4.1.3  Output 

The  articulation  of  the  purpose  of  device  14A2  provides  the  initial 
concept  of  the  training  system.  This  scoping  or  Mroughing  out11  of  the  system 
between  NTDC  personnel  and  the  using  agency  training  personnel  provides 
the  focus  for  the  subsequent  iterations  leading  to  the  decisions  on  the 
functional  characteristics  that  define  the  training  device.  The  design 
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issues  identified  iri  this  initial  analysis  are  further  defined  and  explicated 
in  the  subsequent  analyses  . 

4.2  ORGANIZING  INFORMATION  ON  THE  OPERATIONAL  SYSTEM. 

The  definition  of  training  purpose  provides  the  essential  outlines  for  the 
14A2  training  system.  This  has  established  a basis  for  what  must  be  repre- 
sented in  the  device,  the  magnitude  , of , and  the  types  of  training  required 
and  gross  concepts  about  the  fidelity  of  simulation  in  the  device  hardware. 

o Tixovide s the  basis  for  the  analysis  of  task,  structure  and  the  definition 
of  the  characteristics  of  the  operational  environment  to  be  represented  in 
the  training  system. 

An  analysis  of  ASW  operations  is  now  begun  to  gain  the  necessary 
familiarization  with  those  portions  of  the  operational  system  on  which  the 
training  device  is  based. 

4.2.1  System  Description.  Highlights  of  the  system  descriptive  informa- 

tion pertinent  to  device  design  are  outlined  below.  Greater  detail  within 
categories  is  not  shown,  but  is  suggested  by  the  organization  of  the  information 

4 . 2 . 1 . 1 ASW  Missions 

The  range  of  ASW  destroyer  missions  involve  the  following: 

. prevention  of  submarine  mission  in  an  area 
(SAU  context- -detection  and  classification  of 
submarine,  killing  or  deterring  submarine) 

. intelligence  gathering,  warning  (large  area 
surveillance,  focal  area  surveillance) 

. protection  of  commercial  shipping  (sanitization 
of  an  area  or  lane,  screening  of  a convoy) 

escort  of  naval  forces  (screening  of  naval  forces , 
sanitization  of  harbor  exits  and  entrances) 

# countering  of  submarine  in  local  area  (screening 
and  deterring  submarine,  blockade) 

With  minor  exceptions,  these  missions  are  applicable  to 

general  or  limited  war  with  or  without  conventional  weapons  for  the  follow- 
ing combinations  of  destroyer  forces: 
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„ destroyer  ox  destroyer  escort 
. DD  and  HS 
. DD  and  VP 
. DD  and  VS 
. DD  and  SS 

. DD  and  SS  and  VP/VS/HS 

To  maximize  the  training  effectiveness  of  device  14A2  , two 
broad  mission  contexts,  in  a "hot"  war  situation  are  defined  as  the  environ- 
ment for  training.  These  are  the  screening  context  and  the  search  and 
attack  unit  (SAU)  context.  The  following  is  the  basic  ASW  paradigm: 


Screen.  Screen  elements  protect  more  vulnerable  units  being 

screened  by  employing  diversionary  or  agressive  measures.  Exercises 
in  the  screen  area  will  deal  with  sustained  ASW  operations  in  or  near  the 
main  body  of  ah  operating  force  as  distinguished  from  SAU  exercises  where:, 
the  destroyers  may  start  out  in  a screen,  but  become  a SAU  as  soon  as  they 
are  detached  for  prosecution  in. an  area  remote  to  the  main  force . Screen 
exercises  will  deal  somewhat  with  position -keeping  and  control  of  search 
aircraft  on  the  screen,  but  more  heavily  with  the  prosecution  of  submarines 
inside  or  very  near  the  screen.  Two  examples  of  the  foregoing  are: 


# Urgent  attack  from  Condition  III  Watch  on  a 
contact  detected  inside  the  screen  or  the  TDZ 
of  the  destroyer 

. Coordinated  search  for  a submarine  suspected 
of  being  inside  the  screen  or  very  near  the 
formation  (ex.:  the  use  of  such  plans  as 

Pumpkin  5 IS,  Beetroot  52Ss  and  Carrot  53S) 

Training  in  such  activities  as  the  initial  configuration  of  screen- 
ng  units,  the  selection  of  units  to  compose  a SAU,  reconfiguration  of  a 
icreen  following  the  departure  of  elements  assigned  to  a SAU,  is  considered 
. function  of  the  screen  commander  or  the  OTC  . As  such,  these  are  higher 
evel  tactical  decisions  now  thought  to  be  inappropriate  for  training  on  the 
4A2.  More  appropriately,  such  tactical  decision-making  training  might 
>etter  be  accommodated  on  a training  device  where  the  portrayal  of  the 
lynamic  geometry  of  multiple  units  is  more  comprehensive,  and  with  ex- 
ensive  and  more  sophisticated  communication  possible  among  greater 
lumbers  of  participating  unit  and  force  commanders  . (Device  14A6 , ASW 
Coordinated  Tactics  Trainer,  provides  this  kind  of  training.) 

Search  and  Attack  Units  (SAU).  SAU  operations  involve  the 
jrosecution  of  a datum  or  contact  by  coordinated  surface/air  forces  , whose 
primary  purpose  is  to  search  and  ’’successfully  engage”  enemy  submarines, 
rhe  training  emphasis  in  Action  Selection  and  Implementation  will  be  on 
SAU  operations  , since  considerable  doctrine  exists  for  SAU  missions  and 
which  can  be  trained  in  the  14A2.  The  compositions  of  SAU's  presently 
considered  for  training  are  grouped  into  three  basic  classes:  one  destroyer 

plus  aircraft;  two  destroyers;  and  two  destroyers  plus  aircraft.  These 
groupings  supply  realistic  operating  requirements  to  be  implemented  in 
Device  14A2.  The  destroyer  complement  of  SAU's  has  been  limited  to  a 
maximum  of  two  ships  based  on  the  operating  forces  suggestion  that  more 
than  two  ships  in  a SAU  may  frequently  impair  overall  coordinated  perfor  - 
mane e , especially  in  the  attack  phase  . 

The  destroyer  complement  of  a SAU  can  originate  from  a variety 
of  screen  formations  (circular,  bentline,  etc.),  which  in  turn  are  part  of 
larger  operating  forces.  The  operating  forces  presently  considered  as  a 
source  of  supply  for  the  destroyers  of  a SAU  are: 

, Hunter  Killer  (HUK)  Force  (Destroyer  and  A/C  SAU's) 

, Strike  Force  (Destroyer-only  SAU's) 

. Merchant/Naval  Convoy  (Destroyer -only  SAU's) 

SAU  operations  begin  when  an  OTC  or  Screen  Commander  desig- 
nates and  dispatches  a destroyer  element  (normally  preceded  by  an  air  ele- 
ment in  the  case  of  a HUK  force)  to  prosecute  a datum  or  contact,  and  en- 
compass the  following  phases  of  operation: 
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distant  transits,  and  rejoining  will  be  mediated  to  conserve  training  time 
and  allow  for  focusing  on  the  more  crucial  search  and  attack  phases. 

4.2.2  Functions  of  ASW  Subteams.  The  major  functions  performed  by 
the  subteams  in  ASW  operations  are  outlined  next. 

SONAR 


Conduct  sonar  search;  use  beam-to-beam  procedures  or  sweeps 
of  search  arcs. 


Detect,  classify,  and  track  a 


target;  provide  current  sonar 


target  range  and  bearing  to  CIC,  TJB  IPlot  and  Bridge. 

Deter;  line  and  report  target  aspect  from  doppler  (audio). 

Detect  and  classify  torpedo  and  submarine  hydrophone 
effects  . • • 


Conduct  Mno  echoes ,f  and  lost  contact  search  procedures. 
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UB  PLOT 


# Pexform  target  motion  analysis  on  sonar  contact 
and  develop  a fire  control  solution  (target  course  and 
speed,  WEP)  . Provide  developing  PCS  information 
to  CIO  and  Bridge. 

. Select  ASROC  or  AWTT  weapon  delivery  system, 
lioming  weapon  (Mk  44,  46  Horning  torpedo)  and 
fire  ASROC  or  AWTT. 

• Provide  aided  tracking  (director  control)  to  sonar 
when  fire  control  solution  is  smooth. 

9 Provide  vectoring  information  for  control  of  DASH 
(back-up  method  to  those  available  in  CIC)  . 

9 Deploy  and  activate  material  countermeasures 
(Fanfare);  provide  required  underwater 
communications  (Gertrude). 

CIC 

m Recommend  coordinated  search  plans  (area  and  lost 

contact),  attack  and  support  methods,  torpedo  counter- 
measures, and  torpedo  evasion  techniques. 

m Monitor  conduct  of  coordinated  search,  recommend 
maneuvers  for  own  — ship  and  surface  support  units  , 

. Control  ASW  Air  Control  Net:  Coordinate  search 

functions  of  VS  aircraft  under  advisory  control; 
vector  HS  aircraft  to  dips  (positive  control)  in 
extended  search  lines  , dip  stations  , and  search 
vectors. 

. Control  use  of  RADAR  and  ECM;  detect,  classify,  and 
track  significant  surface  contacts  (submarine,  snorkel, 
periscope,  friendly  forces);  detect  and  classify  electronic 
emissions. 

. ' ■ . , , i 

. Maintain  a plot  of  own-ship,  own-ship  contact,  surface 
support  units , support  unit  contacts  , guide  . 
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Compute  and/or  Plot: 


- DATUM,  datum  time  (to  include  a fix  on  two 
ECM  bearings,  radar,  sonobuoy,  sonar, 
visual  SOSUS). 

Farthest  on  Position  Circles 
Torpedo  Danger  Area  (TDA) 

ETA  to  TDA 
CPA's 

Torpedo  Danger  Zone  (TDZ) 

Limiting  Lines  of  Approach 
Danger  Zone 
Water  Entry  Point 
Firing  Lane  and  Impact  Area 
Weapon  Run-outs 

Torpedo  Hydrophone  Effects  (bearings) 
Knuckles 

Maintain  display  of  important  tactical  information 
(tailored  to  individual  ship  ne  ids)  on  status  boards  . 


Control  SAU  Cl  and  ASW  Common  Nets;  send,  receive, 
and  disseminate  initial  contact  reports,  SWAP  reports, 
amplifying  reports  and  SITREPS,  weapon  arming  and 
firing  reports  . 


Report  entry  into  TDA,  recommend  appropriate  torpedo 
countermeasures. 


Determine  if  bearing/ range  of  initial  contact  is  "clear"  or 
"foul11  (via  surface  radar) 


Determine  submarine  course  and  speed  from  NC-2 
plotter  or  DRT  plot. 

Dead  reckon  submarine  course  and  give  EP's,  recom- 
mend sonar  search  arcs  when  contact  is  lost.  ; ^ 

Vector  DASH  to  and  from  drop  point;  control  weapon 
drop(s). 

Control  ASW  Air  Control  Net;  conduct  vectored  attacks 
using  HS  or  VS  aircraft  under  positive  control. 


Recommend  maneuvers  to  hold  contact  prior  to  atta-cks 
(keep  contact  out  of  baffles). 
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* Recommend  and/or  concur  weapon  selection  and 
firing  time . 

# Recommend  turn  after  weapon  launch.;  control  ship 
during  post-attack  assessment. 


BRIDGE 


# Exercise  tactical  control  of  own-ship  and  other 
support  units  of  SAU,  while  performing  roles  of 
SAU  Commander  and  SAC. 

* Maneuver  own- ship;  control  course  and  speed  via 
engine  orders  and  rudder  angle  control. 


# Approve  all  search  plans,  attack  and  support  methods, 
torpedo  countermeasures,  and  weapon  launches. 

. Control  PRITAC  and  SAUTAC  Nets;  transmit  initial 
contact  report  (PRITAC)  and  maneuvering  orders  to 
other  ships  of  SAU  (SAUTAC). 

# Assure  ship  safety. 

LCCS 


m Approve  all  weapons  launches. 

# Provide  backup  firing  capability  in  case  of  malfunction. 

4.2.3  Man-Machine  Interactions 

4.2. 3.1  ASW  Stations  and  Operator  Duties 

A total  of  twenty-two  personnel  are  involved  in  the  ASW  opera 
tions  of  concern  for  device  14A2  . These  represent  the  essential  manning 
requirements  for  the  UB  plot.  Sonar,  CIC , Bridge  and  LCCS  subteam  com- 
partments to  accomplish  the  range  of  coordinated  training  desired  in  the 
14A2. 


CIC  Compartment 

Personnel  Duties 

Evaluator  Supervises  the  NC-2  and  DRT  plotting 

functions  , evaluates  contact  from  the 

plot.  r-;t  ■ V '■ : 
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Perse  nel 


Evaluator 
(Cont . ) 


CIC  Officer 


No.  1 - Plotter 


Duties 


Supervises  CIC  external  Voice  com- 
munications, and  disseminates  evalua- 
tion and  progress  of  attack. 

Prescribes  search  arcs  to  Sonar  or 
recommended  search  plan  to  the  Bridge 
as  required  (IJS  circuit). 

Keeps  Bridge  informed  of  tactical  situa- 
tion and  makes  appropriate  rec ommenda 
tions  for  the  employment  of  own- ship 
and  sensors . 

Recommends  appropriate  search  plans 
and  employment  of  other  units  when 
operating  in  multi -unit  context. 

Provides  Bridge  with  maneuvering  in- 
formation when  designated  as  assisting 
ship. 

Controls  the  radiotelephone  nets  for 
rapid  communications  with  other  ASW 
units. 

Supervises  the  communications  with 
aircraft. 

Recommends  aircraft  employment  to 
the  CO. 

Supervises  the  conduct  of  VECTACS 
and  MADVECS  and  other  aircraft 
operations  in  accordance  with  existing 
instructions. 

Functions  as  assistant  evaluator. 

Maintains  plot  of  contact  and  own- ship 
on  NC-2/DRT  plot. 

Receives  sonar  tracking  (range  and 
bearing)  information  (61 JS  circuit)  . 
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Personnel 


Duties 


No.  2 - Plotter  Maintains  plot  of  assisting  ships  and 

aircraft  and  of  their  contacts  on  NC-2/ 
DRT  plot. 

Mans  the  radar  information  circuit 
(22JS),  receives  and  plots  range  and 
bearing  information  on  surface  contacts 
being  tracked. 

Surface  Radarman  Detects,  classifies  surface  contacts. 

Provides  tracking  information  (range 
and  bearing)  on  surface  contacts  over 
22 JS  circuit. 

Determines  if  bearing/ range  of  initial 
contact,  “clear"  or  flfoul.  “ 

ECM  Operator  Detects  and  classifies  electronic 

emissions. 

Radar  Supervisor  Supervises  the  operation  of  radar. 

Provides  assistance  as  required  by 
Evaluator  in  areas  of  external  com- 
munications and  plotting  functions 
(NC-2/DRT). 

Mans  ASW  Air  Control  Net  and 
controls  VS  and  HS  aircraft  as 
directed,  to  include  the  routine  \ 
control  of  HS  dip  cycles,  advisoryV^ 
control  of  VS  search  operations, 

HS  and  VS  VECTAC's. 

Plots  significant  ASW  tactical  in- 
formation as  required  by  the  CO 
and  the  Evaluator. 

TJB  / Plot  C ompa  r tment 

ASW  Officer  Verifies  target  data  by  monitoring 

sonar  azimuth  and  range  indicator  for 
comparison  with  geographic  plotter  of 
attack  console. 

. . 
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Personnel 


ASW  Officer 
(Cont.) 


Geographic 

Plotter 


Dutie  s 


Reports  solution  of  target  classifica- 
tion course  and  speed  to  Bridge  and  CIC 
and  compares  with  their  solution. 

Responsible  for  control  of  Fanfare. 

Directs  the  attack  operation  of  all 
manned  ASROC  components  (via 
1JS  circuit). 

Coordinates  missile  selection  with  the 
CO. 


Directs  firing  of  the  ASROC  missile 
when  authorized  by  the  CO. 

Controls  all  ASW  stations  from  UB 
plot. 

Ensures  that  contact  reports  and  classi- 
fication are  promptly  and  properly 
disseminated. 

Recommends  courses  of  action  to 
execute  the  ASW  attack  and  the  ord- 
nance to  be  used. 

Mans  the  1JS  phone  circuit. 

Operates  the  geographic  plotter  sections 
of  attack  director  (A/D). 

Determines  the  submarine’s  course, 
speed,  target  angle,  range  rate,  firing 
time,  weapon  train,  and  other  data 
important  to  the  attack  and  to  ship 
safety . 

Directs  and  monitors  attack  console 
operations  as  ordered  by  the  ASW 
officer. 

Controls  aided  tracking  and  position 
keeping  tracking  in  conjunction  with  sonar. 
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Personnel  Duties 


Geographic  Monitors  digital  computers  and  provides 

Plotter  (Cont.)  input  data  necessary  for  fire  control 

solution . 


Firing  Panel  Controls  weapons  station(s)  (via  JA  or 

Operator  8 JP  circuit)  and  informs  the  ASW  officer 

of  its  status. 


Controls  the  use  of  safety  devices  at 
weapons  stations  . 

Operates  the  firing  control  panel  to 
fire  the  designated  weapon(s)  (ASROC, 
AWTT) 


Sonar  Compartment 

Sonar  Supervisor  Assists  the  ASW  officer  in  supervision  of 

sonar  control* 

Provides  the  ASW  officer  with  continuous 
contact  evaluation. 

Responsible  for  proper  sonar  search 
procedures  and  evaluation  of  sonar 
information . 

Reports  contact  information  over  1JS 
circuit. 

Sonar  Operator  Conducts  sonar  search  as  specified  by 

CIC  using  standard  doctrine  and 
procedures . 

Reports  all  initial  contacts  to  ASW 
stations  via  29  MC  * 

Makes  preliminary  contact  evaluation 
and  classification. 

Tracks  all  contacts . 

Reports  any  evidence  of  torpedoes  or 
submarine  countermeasures. 

O 

ERIC 


v v#  *• 

* 

- 443  - 


NAVTRADEVCEN  69-C-0298-1 


Personnel 

Duties 

Standby  Sonar 
Operator 

Assists  the  search  operator  as 
required. 

Relays  all  information  from  the 
search  operator  to  the  Bridge,  CIC, 
and  UB  plot  (via  61JS  circuit). 

Relieves  the  search  operator,  as 
required,  to  alleviate  strain  of  con- 
tinuous concentrated  effort. 

Bridge  Compartment 


C ommandin  g 
Officer3** 

Responsible  for  all  ASW  procedures 
and  tactics,  approves  tactical 
recommendations  . 

Supervises  the  coordination  with  other 
ASW  units,  and  ordering  of  coordinated 
search  and  attack  plans. 

Approves  weapon  selection. 

Directs  sonar  employment. 

Makes  final  decision  on  target 
clas  s ification . 

Responsible  for  own-ship  safety. 

Officer  of  the 
Deck  (OOD) 

Verifies  that  all  stations  are  properly 
manned. 

Visually  checks  contact  bearing  and 
reports,  ,rbear ing  clear  or  foul . 11 

Compares  target  characteristics  from 
sonar,  UB/  plot  and  CIC  . 

Shifts  control  between  Bridge,  UB  plot 
and  CIC  . 

♦Listing  the  CO  as  part  of  the  Bridge  complement  does  not  imply  that  the 
CO  should  be  physically  located  there  throughout  a training  exercise. 
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Personnel Duties 

Officer  of  the  Issues  orders  for  own-ship  maneuvers. 

Deck  (OOD) 

(Cont.  ) 

JOOD  Assists  the  OOD  and  CO. 

Monitors  RT  circuits. 

Mans  the  1JS  circuit  during  ASW  action. 

Assumes  the  conn  if  so  ordered. 

Surface  Rada r man  Monitors  positions  of  surface  support 

units  . 

RT  Talker  Maintains  communications  between 

own- ship  and  other  support  units 
(PRITAC,  SAUTAC , Screen  Common 
Nets)  . 

Helmsman  Controls  ship's  course  and  speed  via 

rudder  angle  and  engine  orders  . 

T auncher  Captain's  Control  Station  Compartment 

Console  Operator  Operates  the  LCCP  Mark  199. 

Ensures  proper  system  attack  progression. 

Monitors  operation  of  the  relay  trans- 
mitter, Mark  43. 

Reports  detected  malfunctions  to  ASW 
officer  (via  8JP  circuit). 

Verifies  that  all  inputs  have  reached 
launcher  and  that  launcher  is  operating 
properly . 

Responsible  for  weapon  launcher  in  the 
event  of  failure  at  the  Attack  Director 
Console  . 


O 
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4.2. 3.2  Sequence  of  Operations  in  the  System  . An  operational  sequence 
diagram  is  used  as  a means  to  graphically  present  the  interactions  of  men 
and  equipment  in  the  system  operating  sequence.  Table  35  presents  the 
sequence  of  operations  for  destroyer  ASW  functions.  The  most  complex 
mission  environment  is  depicted  (SAU  operations)  to  show  the  relationships 
among  the  major  operating  compartments  of  CIC,  UB/Plot,  Sonar  and 
Bridge. 

4.2.4  Sources  of  Information 

* System  observation  and  discussions  with  knowledgeable  operating 
personnel 

* System  Documentation  (official  Navy  Publications) 

(Confidential) 


- NWP-24-C 

- NWIP-24-  1(A} 

- ATP- 1(A) 

- A TP-28 

- ASAC 

- NAVPERS  10140 

- NAVPERS  10131 

- NAVPERS  10 147 -C 

- NAVPERS  10 144- A 

- NAVPERS  10823-B 


ASW  Operations 
Antisubmarine  Classification 
Manual 

Allied  Maneuvering  Instructions 

Allied  ASW  Manual 

Handbook,  antisubmarine  Control, 

COMASWFORLANT 

Sonar  Technician  1st  and  Chief 

Sonarman  3rd  and  2nd 

Radarman  1st  and  Chief 

Radarman  3rd  and  2nd 

Combat  Information  Center  Officer 


4.3  ANALYSIS  OF  TASKS  INVOLVING  HUMAN  PARTICIPATION.  Effort 
is  now  devoted  to  identifying  and  organizing  the  activities  involved  in  sub- 
team and  team  performance  in  order  to  make  decisions  about  characteristics 
of  the  training  system.  This  is  done  to  the  level  of  detail  needed  to  define 
the  simulation  elements  that  should  be  represented  by  the  device  to  achieve 
the  purpose  of  training. 


4.3.  1 Operational  Structure.  Task  structure  is  developed  from  the  sys- 
tem analysis  data  and  from  the  major  ASW  functions  performed.  It  centers 
on  the  key  ASW  positions  where  the  highest  density  of  interactions  are  in- 
volved^. Satisfying  the  performance  requirements  in  the  CIC,  Sonar  and 
UB/Plot  compartments  and  to  a lesser  extent  in  the  LCCS  and  Bridge  com- 
partments, has  the  greatest  impact  on  design. 


For  the  purpose  of  this  report,  task  analyses  are  presented  below 
for  two  of  the  key  subteams:  CIC  and  Sonar.  Similar  analyses  would  be 

conducted  for  UB/Plot,  Bridge  and  LCCS  compartments,  but  are  not  shown 
here.  Tables  36  and  37  are  CIC  analyses;  Table  38  is  the  Sonar  analysis. 
The  analyses  consider  the  most  complex  mission,  that  of  SAU  operations. 
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TABLE  35. 


OPERATION  SEQUENCE  DIAGRAM 
FOR  DESTROYER  ASW  FUNCTIONS 
INVOLVED  IN  A SEARCH  AND  ATTACK 
UNIT  (SAU)  OPERATION 
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ff 
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As  sumptions  : 

1.  ) A destroyer  type  (e.  g.  a FRAM  I)  with  an  SQS-23 
Sonar.,  MK114  Fire  Control  System,  and  ASROC/ 
DASH/AWTT  weapon  delivery  systems. 

2-  ) A hot  war  situation  exists 


LEGEND 


Information  Received 


□ — Action  Performed 


— Determination/Decision 
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SEARCH  AND  ATTACK  UNIT  (SAU)  OPERATION 
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TABLE  35.  OPERATION  SEQUENCE  DIAGRAM  FOR  DESTROYER  ASW  FUNCTIONS  INVOLVED  IN  A 
SEARCH  AND  ATTACK  UNIT  (SAU)  OPERATION  (Continued) 
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SEARCH  AND  ATTACK  UNIT  (SAU)  OPERATION  (Continued)  
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TABLE  35.  OPERATION  SEQUENCE  DIAGRAM  FOR  DESTROYER  ASW  FUNCTIONS  INVOLVED  IN  A 
SEARCH  AND  ATTACK  UNIT  (SAU)  OPERATION  (Continued) 


NAVTR ADEVCEN  69-C-0298-1 


H 

O 

P 

ft 

m 

p 


rt 

u 

•«-J 

U 


8 

aP 

Jh 

O 

d 

d 

<% 

4_> 

(0 

too 

JU| 

TJ 

(0 

(0 

t>. 
1 1 

rt 

4-1 

CL 

co 

o p o 


o 


<5 

o 

CO 


u 

cl 

d 

O 

C! 

C! 

< 


i 

?H 

«5 


T7T 

d rt 

E W)  i> 
2 © £ 
JL  0) 
CJ  \ 4J 

to  o)  a 
-u>  bo  ® *H 

5 5 too  jl 
O 2 S3  d 

CJ  fn  — i 


CO 

ha 

vO 


U 

n 

O 


6 


Jh 

d 

d 

o 

CO 


O 

r— I 

CL 

0) 

4J 

rt 

CL 

d 

9 

5 U 

rt  rt 

• «-*  4-1 

a g 
5 2 


CO 


O-n  6 


co 

•"S 


w 

o 

Q 

ft 

« 


£ 


o 


CO 

^ o 

a 

d 

o 

a 

a 

< 


a co 
**  rt 


m ^ ~ 
« « « 

a a.  >, 
<2  o.  « 

u t)  i 


CO 

1-3 


a) 

to 

Jh 

o 

u 

4-> 

a; 

too 

?H 

rt 


a 

a> 

d 

cr 

<D 

*+H 

a) 

CJ 

a 

d 

o 

a 

a 

<< 


a 

o 

co 

■H 

Jh 

rt 

CL 

a 

o 

u 

Li 

O 


If. 


CM 

l 

U 

£ 

S 

o 

Jh 

Si  pq 

8 £ 
CL  *£ 
co  > 


6 


0 

r-H 

CL 

CM 

1 

u 

25 


u 

0) 

CL 

W 

rt 

a) 

u 

a 

cj 

o 

a 

d 

< 


u 

0) 

-H 

CL 

CL 

O 

T3 

Jh 

rt 

d 

o 

w 


0) 

i-rt  ,_H 

-=J  & 

O 

CO 

o o 

^ CL 


co 

Jh 

0) 

> 

d 

0) 

9 

a 

co 

O 

T3 

CJ 

0) 

a 

a 

o 

u 

0) 

05 


<D 

too 

d 

rt 

Li 

a 

3 

a 


co 

1-3 


0-Q  CD 


CO 


co 

1-3 


6 6 6 6 6 


0) 


a?  ^ 

Q -S 


<u 


05  o 


(0 


( •;uod)  'SSSiSSV  IV3HHI/  ‘ IN3QI/SSVT  D /ID VXMOD 


HDVIiXV 


O 

ERIC 


£V  C?  *,« 


- 452 


NAVTRADEVCEN  69-C-0298-1 


TABLE  35.  OPERATION  SEQUENCE  DIAGRAM  FOR  DESTROYER  ASW  FUNCTIONS  INVOLVED  IN  A 
SEAUCH  AND  ATTACK  UNIT  (SAU)  OPERATION  (Continued) 
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TABLE  35.  OPERATION  SEQUENCE  DIAGRAM  FOR  DESTROYER  ASW  FUNCTIONS  INVOLVED  IN  A 
SEARCH  AND  ATTACK  UNIT  (SAU)  OPERATION  (Continued) 
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TABLE  36.  ANALYSIS  OF  CIC  TASKS  IN  SAU  MISSION  CONTEXT1 
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We  were  unable  to  locate  any  formally  generated  task  analyses  for  the  14A2  device,  hence  the  task 
analyses  were  accomplished  by  Dunlap  and  Associates,  Inc. 
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4.3.  1.  1 Task  Structure  Summary.  The  task  activities  for  device  14A2 
training  emphasize  skills  and  knowledges  dealing  with  individual  and  team 
procedures  and  with  decision-making  behavior.  A premium  is  placed  on 
the  following  skill  and  knowledge  requirements: 

. Knowledge  of  ASW  procedures  in  operating  individual 
equipment  components  and  in  sub  team  procedures  in 
employing  the  ASW  platform. 

* Knowledge  of  ASW  doctrine  (procedures  and  tactics) 
as  prescribed  in  official  Navy  publications  (e.g.  , 

NWP-24C,  ATP-l-(A),  Vols . I,  II,  ATP-28,  ASAC 
handbook,  etc.). 

# Procedural  skills  in  operating  equipments  in  the  CIC  , 
UB/Plot  and  Sonar  compartments. 

* Heavy  requirements  on  team  interaction  and  coordination, 
vis-a-vis  internal  and  external  communications  and 
decision-making  activities. 

# Information  plotting  (e.g.  , NC-2  Plotter,  DRT , status 
boards)  . 

# Information  processing- -formatting  requirements, 

(brevity  codes  , scramble  tables;  preparing  information 
for  dissemination  based  on  own- ship  sensor  performance). 

* Visual-perceptual  demands  - -emphasis  on  display  inter- 
pretation (e.g.  , sonar,  and  attack  director  in  achieving 
fire  control  solutions;  radar)  and  display  monitoring  (e.g.  , 
displays  of  ship  control  equipments,  NC-2  Plotter,  radar, 
sonar) . 

Limited  requirements  are  placed  on  manual  control  skills. 
Manual  control  requirements  are  most  prominent  in  the  control  of  DASH 
and  in  cursor  manipulation  in  Sonar,  Radar  and  Attack  Director  (UB/Plot). 

Vehicle  control  requirements  are  minimal.  Steering  the  ship 
from  the  bridge  (ordered  speed,  rudder  angle,  and  course)  is  accomplished 
simply  via  a potentiometer  control  rather  than  with  the  controls  found 
aboard  ship. 
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4.3.2  Training  Objectives.  The  training  objectives  reflect  a variety  of 
team  training  situations  ranging  from  drills  on  basic  attack  procedures 
through  complex  situations  involving  coordinated  multi-unit  operations  to 
cover  the  cross-section  of  ASW  operations  in  a "hot  war"  environment. 

The  training  objectives  provide  the  inputs  for  identifying  design  capabilities 
that  must  be  present  in  the  device  in  order  to  accomplish  the  training  deemed 
necessary.  The  training  objectives  point  up  also  the  flexibility  needed  in 
device  capabilities  to  enable  the  instructor  staff  to  select  exercises  for 
training  according  to: 

. Team  skill  level  upon  entering  training 

. Team  learning  rate  during  a training  session 

. Time  available  for  training 

. Specific  training  desired  by  a team  (e.g.,  preparation 
for  an  upcoming  fleet  exercise). 

Several  levels  of  team  training  are  envisaged,  each  having  specific 
and  well-defined  training  objectives  . These  levels,  described  next,  repre- 
sent a build-up  of  exercise  difficulty  and  provide  the  rudiments  for  an  in- 
structional package  beginning  with  the  entry  level  of  a team  and  progressing 
through  advanced  ASW  tactical  employment  of  own-ship  in  a SAU  context. 

4.3.2. 1 Basic  Attack  Exercises.  These  repre sent  initial  training  exer- 
cises and  cover  the  basic  attack  procedures  associated  with  own-ship  weapons 
systems.  The  team  is  required  to  practice  developing  fire  control  solutions 
and  launching  the  weapons  they  possess  against  targets  they  have  searched 
for  and  detected.  Exercises  start  with  an  undetected  target,  followed  by 
early  detection.  The  team  briefing  prior  to  the  exercise  instructs  the  team 
to  develop  a fire  control  solution  and  to  launch  the  weapon  designated  m the 
exercise  as  rapidly  and  accurately  as  possible.  The  exercises  use  the  one- 
destroyer  (own-ship)  versus  one-submarine  situation.  Exercises  should  be 
provided  which  specify  ASROC , AWTT,  DASH,  RTDC  (nuclear)  and 
HS/VECTAC  attacks. 

The  training  objectives  stress  the  fundamentals  of  ASW  opera- 
tions in  subteam  and  team  performance.  They  center  on  the  development  of 
coordinated  procedures  and  internal  communications  in  achieving  fire  control 
solutions  and  launch  of  the  available  weapons  systems.  In  addition,  the  team 
is  exposed  to  basic  submarine  tactics,  torpedo  hydrophone  effects,  equipment 
casualty  and  the  geometric  aspects  of  threat,  including  maneuvering  for 
attack  and  lost  contact  procedures.  The  specific  concerns  include: 

Equipment  setup  and  checkout  procedures  in  the 
subteam  compartments 
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• Appropriate  utilization  of  internal  com  . jUiiication  circuits 

• Procedures  in  search,  and  detection 

• Procedures  for  readying  weapons  systems  for  attack 

• Prosecution  of  targets. 

4.3.  2.1.  1 Training  Objectives.  Conduct  accurate  and  timely  ASROC 

attacks  and  AWTT  attacks  using  Mk  44  and  Mk  46  torpedoes  against  straight 
running  and  maneuvering  targets  at  various  ranges  and  courses  relative  to 
own-ship.  Implement  standard  procedures  for  search,  detection,  fire  control 
and  launch.  Implement  lost  contact  procedures. 

Sonar 


• Conduct  standard  no  echo  and  sector  searches 

• Provide  timely  detection  and  target  status  reports 

• Track  targets  accurately  including  difference  and 
director  modes 

• Employ  post  attack  doctrine  for  Mk  44-1  torpedo. 
UB  Plot 

• Perform  target  motion  analysis 

« Conduct  weapon  selection  and  launch  sequence 

• Monitor/ recommend  ship  control  during  attack 

• Identify  and  rectify  malfunctions  . 


CIC 

• Monitor/recommend  sonar  search 

• Monitor/ recommend  ship  control 

• Monitor/ verify  fire  control  solutions 

• Implement  tactical  deception  techniques 

• Plot  own-ship  target  and  weapon  positions 

• Implement  torpedo  evasion  tactics. 

LCC  S 

• Monitor  weapon  launch  sequence 

• Take  corrective  action  when  malfunction  occurs. 
CONN 

• Monitor  fire  control  solution  (ASROC)  and  approve 
weapon  selection 

• CONN  ship  to  retain  contact,  minimize  vulnerability, 
conduct  attack  at  standoff  ranges  . 
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4.  3.2.2  Training  in  Screening  Operations,  These  are  more  difficult 

exercises  in  which  the  team  performs  ASW  operations  where  the  mission 
is  to  protect  a screened  unit.  The  purpose  is  to  develop  an  understanding 
of  imminent  threat  to  units  in  the  formation  and  the  requirements  for  rapid 
response  to  this  threat.  The  training  situations  require  quick  employment 
of  various  weapons  systems  from  the  screen  position  with  minimal  maneuver- 
ing after  contact.  The  weapons  systems  include  ASROC  , AWTT  and  aircraft 
already  under  positive  or  advisory  control.  Weapon  launches  must  be  safely 
accomplished  in  the  presence  of  other  friendly  units. 

4. 3. 2. 2.1  Training  Objectives 

# Introduce  the  team  to  the  principles  and  procedures 

of  screening  to  include:  crossover  patrol  of  a harbor 

entrance  and  shallow  water  operations,  station-keeping 
in  a sector  screen,  screening  other  units  by  means  of 
a bentline  screen,  screen  penetration  by  submarine, 
station-keeping  in  a sector  screen  about  a harbor 
entrance,  shallow  water  procedures  . 

# Establish  and  maintain  positive  control  over  aircraft, 
demonstrate  advisory /positive  control  of  HS  or  VS 
Aircraft  in  sector  screen  stations. 

# Introduce  the  team  to  coordinated  search  and  attack. 

# Conduct  urgent  or  quick  attacks  from  Condition  IAS 
and  Condition  II  on  targets  of  immediate  threat  to  the 
force  using  the  best  available  weapon  system. 

Specific  requirements  within  these  overall  objectives 
include  the  following: 

# Accomplish  external  reporting  procedures  to  the  OTC 
(including  initial  Contact  reports,  Amplifying  reports. 
Warning  reports  and  Firing  reports,  via  plain  language 
brevity  code  and  the  scramble  table)  . 

# Compute  and  plot  all  information  relevant  to  screening 
operations  (Torpedo  Danger  Zone,  Danger  Zone, 
Limiting  Lines  of  Approach,  submarine's  most 
dangerous  cone  of  courses)  . 

# Employ  sonar  in  the  situation  where  mutual  interfer- 
ence occurs  among  units.. 
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• Conduct  urgent  attacks  on  targets  of  immediate 
threat  to  the  force. 

• Achieve  VECTACs  with  aircraft  already  under  posi- 
tive control  in  sector  screen  stations. 

» Employ  weapons  including  weapon  selection,  decisions 
on  when  to  fire,  and  reattack. 

• Continue  training  in  the  development  of  fire  control 
solutions  and  launch  procedures  (but  now  in  the 
presence  of  friendly  units). 

4. 3. 2. 3 Training  in  SAU  Operations.  Training  in  SAU  operations  where 

the  team  is  in  control  of  surface  and/or  airborne  units  is  the  most  Complex 
and  difficult  level  of  operations  and  involves  sequences  whereby  the  team 
applies  the  procedures  and  doctrine  in  coordinated  approach  to  the  scene-of- 
action,  and  coordinated  search  and  attack.  The  exercises  present  the  funda- 
mentals that  will  prepare  a team  to  better  handle  the  inevitable  variations  in 
coordinated  search  and  attack  operations  that  can  occur  in  the  ASW  environment. 


4. 3. 2. 3.1  Training  Objectives.  Implement  the  following  activities  and 

dependent  supporting  procedures  in  conducting  single  and  multiple  weapons 
attack  in  coordinated  SAU  operations  against  conventional/ nuclear  submarines 
of  various  characteristics  (speed,  depth,  maneuvering)  in  various  ocean 
environments  . 


Execute  procedures  in  approach  to  the  scene  - of-action . 
This  includes  the  selection/ implementation  of  approach, 
countermeasures  employment  and  steering. 

Achieve  computation  and  plotting  requirements  (CPA 
to  Datum,  ETA  to  TDA,  Time  Late  to  Datum,  Furthest- 
on-Position  Circle,  Weapon  Firing  Lane  s . 

Develop/implement  coordinated  search  plans  at  the 
scene- of- action.  This  includes  area,  intercept,  and 
lost  contact  search  plan  procedures  (e.g.  , OAK  TREE 
variations,  ACORN  (lost  contact),  PINEAPPLE). 

Develop/implement  coordinated  attack  plans  and  pro- 
cedures at  the  scene-of-action  (e  . g.  , Geo-Sector  , 
Lock-on,  Deep-Creep  plans;  ASROC , AWTT,  DASH, 
Helicopter  and  fixed-wing  VECTACS;  own- ship  and 
assist  ship  interactions;  single  or  multiple  weapons 
attack  and  reattack). 
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m Conduct  reporting  procedures  between  own- 
ship  and  the  OTC  (SAU  Cl  and  PRITAC 
Nets)  including  Contact  reports  , Amplify- 
ing reports,  and  Weapon  Warning  and  Firing 
reports;  and  reporting  procedures  between 
own-ship  and  support  units  (SAUTAC  Net). 

4.4  GROSS  DEVICE  DEFINITION.  The  boundaries  of  device  14A2 
can  now  be  defined  in  terms  of  gross  hardware  orientation.  This  cen- 
ters on  the  trainee  compartment  requirements  and  represents  the 
correlation  between  that  piece  of  the  operational  universe  to  be  simu- 
lated and  the  ASW  operational  environment. 

4.4.1  ASW  Equipment  Requirements  for  Device  14A2 

a.  Underwater  Battery  Plot  and  Sonar 


. Mk  53  Attack  Console  (Mk  114  FCS) 

. Mk  38  Attack  Console  (Mk  111  FCS) 

(for  ships  having  this  system; 
note — the  Mk  111FCS  is  in  the 
14A2A;  it  is  not  in  the  14A2B) 

• Mk  143  Position  Keeper  Computer 

. AN/SQS-23  Sonar 

. Torpedo  Countermeasure  Equipment  (Fanfare) 
. Ship's  instruments 
, Communication  equipment 
b.  Combat  Information  Center 
. NC-2  Plotting  System 
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. Dead  Reckoning  Tracer  Mk  6 

. Radar  Plan  Position  Indicators  (2) 

. DASH  controller  station 

, Plotting  and  status  boards 

, AN/WSA-1  control  equipment 

, Ships  1 Instruments 

, Communication  equipment 

c.  Conning  Station 

, Own -Ship  Helm 

. Radar  plan  position  indicator 

• Position  indicator  Mk  78 

• Ship's  instruments 

. Communication  equipment 

d.  Launcher  Captain's  Control  Station 

. Launcher  Captain's  Control  Panel  Mk  199 
. Relay  Transmitter  Mk  43 

• Communication  equipment 

4.4.2  Overall  Device  Layout.  The  overall  layout  of  the  compartments 
comprising  the  device  is  shown  in  Figure  47.  The  block  layout  of  the  major 
equipment  complexes  for  UB /Plot- Sonar  , CIC,  LCCS,  and  the  Bridge  are 
shown  in  Figures  48,  49  and  50. 

4.4.3  Operational  Units  and  the  Tactical  Area  Employed.  Eight  vehicle 
units  are  provided  to  handle  the  range  of  complex  tactical  situations  involved 
in  screen  and  SAU  operations.  These  are: 

, Own- ship 
• Support  vehicle  1 

. Support  vehicle  2 (Note:  a support  vehicle  may  be  used 

as  an  assist  ship  in  a 2-destroyer  SAU  operation;  two 
support  vehicles  may  be  used  as  destroyers  in  a screen 
in  support  of  own- ship) 
v Aircraft  1 
. Aircraft  2 

. Aircraft  3 (Note:  1 aircraft  unit  is  employed  as  DASH 

where  applicable) 

^Based  on  the  "Operator's  Guide  for  Surface  Ship  ASW  Attack  Training  Devic 
14A2B  , " NAVSO  P3169  (U),  U.S.,  Naval  Training  Device  Center,  Orlando, 
Florida,  July  1968  (Conf.) 
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Figure  47.  Overall  Device  Layout  Device  14A2 
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Figure  48.  Underwater  Battery  Plot  and  Sonar 
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Figure  49-  Combat  Information  Center 


Figure  50.  Launcher  Captain  Control  Station  (LCCS)  and 
Bridge/Conning  Station 
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# Submarine  1 

. Submarine  2 (Note:  submarine  2 may  be  used  as  an 

aircraft  carrier  in  applicable  missions) 

The  tactical  area  of  the  engagement  ranges  from  a 32  x 32  mile 
area  to  a 256  x 256  mile  area  in  the  following  increments: 


32  x 32  miles 
64  x 64  miles 
128  x 128  miles 
2 56  x 2 56  miles 


4 4 4 Characteristics  of  the  Trainee  Population.  Each  team  in  the  14A2 

is  composed  of  trainees  that  are  members  of  an  ASROC  ship's  ASW  comple- 
ment. It  is  assumed  that  each  team  member  is  already  qualified  in  the  billet 
for  the  training  (note:  while  this  is  not  operationally  feasible  in  every  instance 

the  attempt  is  made  to  send  trainees  qualified  in  the  billets  to  the  14A2  device) 


4.4.5  Scheduling  of  Training.  The  flexibility  of  the  device  to  provide  a 

wide  variety  of  ASW  tactical  situations  makes  it  admirably  suited  to  a course 
of  instruction  ranging  from  relatively  simple  one-destroyer  on  one-submarine 
procedural  team  training  (detection,  fire  control  solution,  weapon  selection 
and  launch)  through  complex  multi-unit  search  and  attack  operations  (SAU 
contexts).  Thus,  a complete  course  of  instruction,  employing  aperies  of 
training  exercises  graduated  in  difficulty  may  be  accomplished.  (Note:  At 

present,  teams  are  scheduled  for  14A2  training  when  the  ship  is  in  port.  The 
most  prevalent  schedule  is  for  each  ship  to  send  a team  for  two  consecutive 
days  of  training,  once  every  three  months). 

4.4.6  Output.  The  14A2  training  system  in  its  gross  specifications  is 
firmed  up  at  this  point,  and  from  here,  the  detailed  characteristics  of  the 
operational  environment  to  be  simulated  for  training  are  developed.  In  the 
normal  scheme  of  development,  the  process  of  defining  the  device  character- 
istics is  iterative,  beginning  with  the  gross  outline  of  the  training  system  and 
continuing  in  refinement  and  modification  as  additional  technical  and  adminis- 
trative inputs  are  organized,  until  the  functional  description  of  the  training 
device  is  complete. 


XA  Utilization  Guide  for  14A2  team  training  is  presently  in  existence: 
Dunlap  and  Associates,  Inc.,  "Device  14A2  Series  Utilization  Guide  for 
Surface  Ship  ASW  Attack  Team  Training."  (U),  Four  Volumes,  BSD 
69-710,  May  1969.  (Conf.) 
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4.5  CHARACTERISTICS  OF  THE  OPERATIONAL  ENVIRONMENT  TO 
SIMULATE  FOR  TRAINING.  The  analyses  thus  far  have  provided  the  basis 
for  defining  that  portion  of  the  operational  system  to  be  simulated  and  or- 
ganized as  a training  system.  A number  of  key  decisions  have  been  made: 

• Purpose  of  the  14A2  training  device  articulated  and  the 
simulation  requirements  described. 

• Training  objectives  identified  and  organized. 

• Task  structure  defined  and  organized. 

• Equipment  complexes  identified  and  the  fixed  components 
set  forth. 

• Overall  training  system  and  operational  modes  roughed  out. 

The  effort  now  concentrates  on  selecting  the  design  alternatives  which 
will  define  the  manner  in  which  the  synthetic  environment  for  training  will  be 
provided  and  how  the  shaping  of  team  learning  (structure  and  control  of  train- 
ing)  will  be  accomplished.  The  human  factors  contribution  to  design  embrace 
two  distinct  phases:  1)  the  fidelity  of  simulation  issues  in  representing  the 

operational  environment  in  the  trainee  compartments  , and  2)  the  management 
of  training  which  concerns  the  structuring,  controlling  and  monitoring  of 
training  at  the  instructor  station. 

Note:  How  the  device  was  finally  configured  (e.  g.  , Device 

14A2A,  14A2B)  is  well  known  in  its  details  . What  we  will 
do  here  is  highlight  the  points  for  decision  within  the  two 
quite  different  but  interrelated  categories  of  trainee  com- 
partment and  instructor  station  to  reflect  the  design  options 
that  have  human  factors  implications  and  affect  training 
capability. 

4.5.1  Representing  the  Operational  Environment  in  the  Trainee 

Compartments  . So  far  as  organizing  the  environment  in  which  the  learning 
will  take  place  is  concerned,  the  central  issue  is,  fidelity  of  simulation. 

For  human  factors,  fidelity  has  meaning  in  terms  of  the  training  process 
and  the  realism  necessary  to  promote  transfer  of  training.  Defining  the 
design  characteristics  for  maximizing  transfer  of  training  from  the  syn- 
thetic environment  revolves  essentially  about  two  interrelated  questions: 
what/how  much  should  be  simulated?  and  how  well  should  this  be 
represented? 

Trainee  station  design  considers  must  prominently  the  issues  in- 
volved in  achieving  simulation  fidelity,  that  is,  specifying  the  hardware 
(displays/controls)  involved  in  the  device  and  the  fidelity  levels  required 
at  the  man-machine  interfaces,  ranging  from  high  fidelity  in  engineering 
simulation  to  deliberate  departures  from  realism  involving  special  features 
of  design  not  found  in  the  operational  system,  in  order  to  enhance  learning 
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To  place  trainee  compartment  design  in  perspective,  the  range 
of  human  factors  issues  and  design  alternatives  are  subsumed  under  two 
major  categories: 

. Configuration  of  the  synthetic  system 
. Simulation  elements  and  fidelity  of  simulation 


4.  5.  1.  1 Synthetic  System  Configuration.  The  job  at  this  point  is  to 

flesh-out  the  system  components  involved  in  the  trainee  station.  The  need 
for  specific  simulation  capabilities  is  now  explored  based  on  the  groundwork 
laid  in  the  previous  phases.  To  be  resolved,  are  those  portions  of  the  sub- 
systems that  must  be  represented  that  significantly  affect  training  capability. 
Decisions  are  made  on  the  "core”  equipments  needed  to  operate  analogous 
to  the  real  world.  What  instruments /equipments  are  operable  replicas, 
what  are  operable  but  non-replicas  and  what  are  non-operable  replicas 
depend  on  training'  purpose  . 

The  fleshing  out  (i.e.  , defining  what  is  to  be  simulated)  of  the 
configuration  is  described  in  the  following. 


3 
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4. 5.  1 . 1 . 1 Government  Furnished  Equipment  vs  . Simulated  Equipment. 

The  decision  to  use  government  furnished  equipment  vs  . simulated  equipment, 
is  for  the  most  part,  an  engineering  issue,  affected  by  availability  of  the 
equipment,  maintenance  considerations  and  budget/ cost  factors  . 


The  decision  was  made  to  simulate  all  equipments  except 
the  plotting  equipment  (NC-2,  DRT)  and  the  DASH  Control  Console.  Since 
direct  transfer  of  training  is  required  from  14A2  equipment  to  actual  ship- 
board equipment  and  procedural  competency  in  operating  the  equipments  is 
an  initial  requirement,  the  physical  appearances  of  equipments  with  all 
operating  controls  and  indicators  shall  be  the  same  for  the  following: 


AN/SQS-23  sonar 

Attack  director  consoles  (Mk  53,  Mk  38  (as  required)) 
Mk  143  position  keeper  console 
Mk  78  position  indicator 

Mk  199  Launcher  Captain  Control  panel 
Ship  control  equipments 


The  radar  repeaters  are  not  replicas  of  the  operational  radar  , 
for  this  is  discussed  in  Section  4.5. 1.2.2.  ..  ..... 


The  rationale 


Communication  circuits  are  required  as  in  the  operational 


en vir  onme  nt . 
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• Intercom-  - (21  MC,  27  MC,  29  MC) 

e Sound-powered  telephone  (1JS,  JA,  JC, 

61JS,  2 1 JS,  8JP,  5JV) 

. RT  Nets  (PRITAC,  SAUTAC,  screen 

common,  SAU  Cl,  ASW  common,  ASW 
Air  Control  Net)* 

4.  5.  1 . 1 . 2 Real-World  Visual  Attachments  . There  is  no  requirement 

for  outside- the-platform  real-v/orld  visual  representation.  Visual  require- 
ments pertain  only  to  electronic  displays. 

4.5.  1.  1.3  Platform  Motion.  There  is  no  requirement  for  platform 

motion  since  the  training  emphasis  is  on  instruction  in  ASW  team  procedures 
and  tactics  in  Cl C,  UB/Plot  Sonar  and  the  Bridge.  The  Manual  Control  re- 
quirement (controlling  the  ship  from  the  Bridge)  serves  only  to  ensure  that 
own-ship  is  positioned  as  ordered  in  the  tactical  environment;  training  in 
steering  the  vehicle  is  not  a training  objective. 

4.5.  1.  1.4  Sonar  Classification  Simulation.  A significant  design  option 

desirable  for  the  14A2,  concerns  the  simulation  of  the  parameters  permitting 
the  target  classification  task  in  sonar  operations.  The  target  classification 
capability  is  not  represented  in  the  14A2  . The  plausible  reasons  for  omitting 
this  crucial  aspect  of  ASW  are  presumably  tied  in  with  initial  concepts  regard- 
ing the  training  purpose  of  the  device  and  savings  in  money  vis-a-vis  the 
initial  ideas  about  the  purpose  of  the  training  (particularly  since  Device  14A6 
serves  as  a coordinated.  ASW  tactical  trainerj.  It  seems  that  the  device  was 
initially  viewed  as  an  early  attack  weapon  system  trainer  in  which  the  pro- 
cedures and  doctrine  involved  in  achieving  fire  control  solutions  and  launches 
for  ASROC,  A WTT  and  DASH  were  paramount.  This  emphasis  placed  on 
honing  team  coordination  in  the  procedures  and  operations  associated  with 
target  detection,  fire  control  solution  and  weapon  launch  would  make  plausible 
the  decision  to  omit  the  classification  capability  in  sonar  in  view  of  the  engin- 
eering problems  and  associated  costs. 

But  the  omission  of  the  classification  function  interferes 
with  realizing  the  full  potential  of  the  device,  all  other  things  being  equal. 
Classification  (false  target  capability)  introduces  the  element  of  uncertainty 
in  threat  evaluation  requiring  decisions  for  dealing  with  false  targets  and  false 
contacts  and  the  attendant  time  losses  in  determining  if  the  contact  is  a possible 
submarine.  Lack  of  classification  uncertainty  also  modifies  team  procedures 
away  from  the  realism  of  real-world  operations.  Currently,  the  operations 
in  the  device  from  contact  through  fire  control  solution  and  launch,  notwith- 
standing the  need  for  procedural  adequacy , makes  the  process  more  mechanical 
than  desired.  The  presence  of  classification  cues  forces  a greater  decision- 
making requirement  on  the  team  and  greater  tactical  involvement  reflected 
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in  more  difficult  training  exercises,  e.g.  , increased  interactions  among 
own-ship  personnel  and  between  own-ship  and  other  units.  Sonar  subteam 
skills  are  also  enhanced  due  to  the  greater  whole-task  replication  in  the 
device  resulting  in  increased  involvement  and  responsibilities  for  sonarmen. 

Achieving  the  false  target  capability  requires  the  following 
classification  cues  beyond  what  is  currently  in  device  design. 

m Own -ship  wakes  and  knuckles 

. Vehicle  size  or  aspect 

m Submarine  knuckles 

m Non- submarine  returns — (submarine  generated) 

launched  decoys,  false  target  cannisters, 
jammers  (NEA  beacon) 

. Non- submarine  returns  - -(natural) 

phenomena  such  as  whales,  schools  of  fish, 
reefs,  wrecks,  pinnacles. 

4.  5.  1 . 1. 5 Cost  Considerations  Influencing  the  Design  of  14A2  . 

The  specification  of  design  for  a training  system  must  account  for  the  issue 
of  costs  vs.  training  benefits.  Tradeoff  analyses  are  required  to  put  into 
perspective  the  training  value  vis-a-vis  increased  operational  capability 
afforded  by  a design  alternative  per  dollar  spent.  Thus  , cost  is  an  important 
determinant  in  any  ultimate  design. 

We  were  unable  to  assemble  a cogent  story  on  cost- 
effectiveness  tradeoff  considerations  for  device  14A2,  hence  this  issue  is 
not  discussed  here.  The  importance  of  cost  vs.  training  value  analyses 
in  any  design  process  is,  however,  emphasized. 

4,5.  1.2  Simulation  Elements.  With  the  configuration  of  the  device  set, 

the  issues  of  specifying  those  parameters  which  define  nhow  well*1  simulation 
is  achieved,  are  resolved.  Specifying  the  fidelity  of  simulation  is  the  crucial 
requirement  in  representing  the  synthetic  environment  in  which  training  is 
conducted.  The  human  factors  input  concerns  the  definition  of  the  simulation 
elements.  These  represent,  in  hardware  terms,  the  parameters  and  their 
values  (range,  envelope,  number)  needed  to  accomplisn  the -desired  training, 
hence  govern  the  range  and  complexity  of  task  installation  in  the  device.  The 
simulation  elements  which  are  controllable  (manipulated  manually  via  the 
instructor  station)  determine  the  perceptual  equivalence  of  the  training  en- 
vironment and  the  operational  situation.  The  ability  to  provide  the  desired 
training  in  the  identified  tasks  is  directly  dependent  on  the  availability  and 
adequacy  of  these  simulation  elements  in  a training  device  (i.e.  , which  ones 
are  selected  and  how  usefully  they  are  represented).  Shortcomings  in  the 
simulation  of  these  elements  define  the  shortcomings  in  the  training  capa- 
bility  of  a device  (i.e.  , simulation  elements  are  associated  with  the  repre- 
sentativeness / complexity  of  the  tasks  to  be  trained  in  tine  device). 
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Adequate  specification  of  these  elements  is  required  so  as  to 
insure  that  training  purpose  and  training  objectives  are  achieved  (thereby 
enhancing  the  transfer  of  training  potential).  Since  these  elements  are 
manipulatible , they  also  affect  the  excellence  of  the  utilization  context 
for  the  sequencing  of  training  (again,  in  terms  of  purpose  and  training 
objectives).  In  short,  the  simulation  elements  define  what  is  needed  in 
the  task  environment  to  achieve  the  expected  transfer  of  training,  assuming 
effective  utilization  of  the  device. 

These  controllable  elements  refer  to  the  training  mission  en- 
vironment at  the  trainee  compartments  and  narrow  down  to  three  major 
groupings  of  characteristics:  own-vehicle  and  support  units;  submarine 

targets ; and  the  media. 


4.5.  1.2.  1 Levels  of  Fidelity.  Five  levels  of  simulation  fidelity  are 

pertinent  to  the  human  factors  specification  of  the  characteristics  required 
to  insure  effective  task  installation  for  the  14A2  device.  How  well  each 
simulation  element  is  to  be  represented  for  training  should  be  evaluated 
in  terms  of  these  five  levels  or  dimensions.  Table  39  describes  the  fidelity 
of  simulation  levels  used  in  evaluating  each  of  the  simulation  elements 
identified.  For  Level  1,  the  decision  is  to  either  select  or  exclude  the 
element  from  simulation.  Based  on  the  decision  to  represent  an  element  in 
design,  the  following  dimensions  are  examined  to  determine  the  fidelity 
required  to  be  useful  for  training.  For  Level  2,  the  decision  is  to  select 
the  required  value(s)  for  the  element  (for  example,  six  levels  of  sea  state 
for  sonar  operations);  for  Level  3,  the  decision  is  in  terms  of  engineering 
precision  (for  example,  sonar  signals);  for  Level  4,  the  decision  is  in  terms 
of  a necessary  reduction  in  tolerances  , or  a backing  of  from  engineering 
fidelity  (for  example,  false  target  cues  for  sonar  classification);  and  for 
Level  5,  the  decision  deals  with  special  features  of  the  training  system  and 
concerns  the  desirability  of  deliberately  deviating  from  the  engineering 
fidelity  continuum  to  enhance  transfer  effects  (for  example,  employing  a 
stimulus  enhancement  technique  on  a primary  visual  display,  or  displaying 
alpha-numeric  performance  information  (augmented  feedback)  directly  on 
a student’s  CRT  display). 

The  whole  array  of  the  simulation  elements  that  must  be 
evaluated  for  representation  in  the  design  of  the  trainee  station(s)  to  achieve 
control,  display  and  procedural  requirements  , is  presented  next.  These 
elements  of  simulation  are  organised  in  terms  of  own-ship  and  support 
units,  target,  and  media  characteristics.  For  each  class  of  task  element 
within  each  of  these  three  groupings  , decisions  must  be  made  about  various 
dimensions  of  fidelity  required.  To  aid  in  this  determination,  the  five  levels 
for  evaluating  fidelity  of  simulation,  identified  in  Table  39,  are  used.  Thus, 
in  the  following  outline,  the  relevant  classes  of  simulation  elements  are 
grouped  within  major  subsystems  applicable  to  the  device  under  consideration 
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TABUS 

39.  FIDELITY  OF  SIMULATION  LEVELS  FOR 
EVALUATING  SIMULATION  ELEMENTS 

Level 

Description 

i 

Inclusion/ exclusion  of  the  element. 

2 

Representative  envelope/ steps / value  for  the  element. 

3 

Degree  of  fidelity  required  (where  engineering  state- 
of-the-art  is  adequate). 

4 

Fidelity  achievable  where  reduction  in  tolerances  is 
required  (where  engineering  state-of-the-art  is  less 
than  adequate)  . 

5 

Deliberate  departures  from  realism  to  enhance 
training  effectiveness  (deviations  in  configuration/ 
operation  associated  with  the  operational  system 
being  simulated)  . 

This  is  accomplished  (as  applicable)  independently  for  each  of  the  simula- 
tion areas  (own-ship,  target,  media)  since  the  classes  of  simulation  ele- 
ments are  different  for  each  area.  Thus,  for  example,  the  classes  of 
simulation  elements  for  the  sonar  subsystem  are  identified  for  own- ship 
and  support  units,  again  for  target  characteristics,  and  again  for  the 
simulated  medium.  The  appropriate  fidelity  dimension(s)  to  consider  is 
indicated  (identified  by  corresponding  number  in  Table  39)  for  each  class 
of  simulation  element  in  each  subsystem  for  each  of  the  three  simulation 
areas . 

4.  5.  1.2.2  Simulation  Elements  to  Represent  in  Device  14A2 

I.  OWN  VEHICLE  AND  SUPPORT  UNITS 

A.  SONAR  (Electronic  Visual  Symbol  Display) 

1.  OWN- VEHICLE  RETURNS  (fidelity  dimensions  ©CD) 


• own- ship  noise 

(flow  noise  , turbine  noi s e) 

• wake 


appropriate  video/audio 
characteristics;  appropriate 
video  degradation  as  a function 
of  speed  of  increase  correlated 


O 
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» knuckles 

, torpedo  hydrophone  effects 
. countermeasures 


with  optimum  sonar  speed; 
appropriate  audio  degradation 
as  a function  of  speed  increase 
correlated  with  optimum  sonar 
speed. 


Note:  Neither  own-ship  wakes  or  knuckles  are  presently  provided 

in  14A2  . Thus,  no  opportunity  is  provided  CIC  to  plot  or  other- 
wise account  for  the  phenomena  and  to  coordinate  with  sonar  to 
prevent  acquisition  and  tracking  of  these  false  targets. 


Note:  Audio  degradation  is  presently  provided  but  is  not  effective 

t>0low  22  knots  . Screw  beats  are  heard  above  22  knots  and  con- 
tact can  unrealistically  still  be  held  when  own- ship  speed  is  at 
34  knots  . 


• cavitation 


own- ship  cavitation  sounds 
should  be  detectable  when 
cursor  is  placed  in  baffles 
area;  no  video 


2.  SUPPORT  UNIT  RETURNS 


(fidelity  dimensions 


©©> 


support  ship(s) 

(aspect) 

echo-ranging  effects 
(mutual  interference  effects 
from  surface  ships  , dipping 
helos , sonobuoy) 

knuckles 


video:  appropriate  blip  size, 

shape,  brightness  for  the 
vehicle  , correlated  with  aspect 

audio:  appropriate  echo  quality  , 

doppler  effects  correlated  with 
aspect 


9 wakes 

Note:  In  the  present  14A2  device  configuration,  sonar  returns  are 

not  sensitive  to  vehicle  size  or  aspect  which  are  cues  to  classifi- 
cation (classification  is  discussed  in  Paragraph  4.5. 1.2.3.  1)  as 
to  the  effects  of  echo  ranging,  mutual  interference  effects  are 
not  presently  simulated.  These  effects  would  have  training  value 
for  screen  and  SAU  operations  . Neither  knuckles  or  wakes  from 
friendly  forces  are  simulated- -again  these  are  good  sources  of 
false  targets. 
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3.  PPI  DISPLAY  CHARACTERISTICS  (fidelity  dimension  (?)  ) 

• echo  quality 

■ pip  quality  (aspect) 

• axis  angle 

m trace  length 

* differential  range  rate 

4.  VISUAL  DISPLAY  REQUIREMENTS  (fidelity  dimension  (?)  ) 

• acuity 

# luminance 

m refresh  rate  (flicker) 

* contrast  sensitivity 


« resolution 

5.  TRANSFER  OF  SONAR  DATA  TO 

FIRE  CONTROL  SUBSYSTEM  (fidelity  dimension  (?)  ) 

. range,  speed,  bearing,  target 
course  (manual  insert) 


. generated  target  tracking 
RADAR  (Electronic  Visual  Symbol  Display) 

1.  OWN -VEHICLE  RETURNS  (fidelity  dimensions  ©©> 

m reflection  echoes  * 

# wake 

m superstructure  masking  (dead  zones) 

m interference  from  other  radars  in 

own  vehicle  or  in  proximity  to  own  vehicle 

m antenna  height  above  water 


ERJC 
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Occurrence  is  associated  with  vehicle  type.  Appearance 
and  definition  are  correlated  with  envelopes  of  occurrence 
for  vehicle  type  and  speed. 


Note:  Own-ship  returns  are  not  provided  for  in  the 

14A2.  This  is  a design  option  that  would  yield  sources 
of  false  contacts. 


2.  SUPPORT  UNIT  RETURNS 
• surface  ships 

. aircraft  (HS,  VP,  VS,  DASH) 


(fidelity  dimensions  ©©  > 

blips  for  ship  and  aircraft  should 
be  proportional  in  size,  shape 
and  brightness  to  vehicle  type, 
aspect,  range  and  range  scale 
selected 


Note:  Ship  and  aircraft  returns  are  presently  provided  in  the 

device,  however,  blip  size  is  not  propox-tional  to  vehicle  size 
or  aspect.  Maximum  range  is  not  realistic  (limited  radar 
horizon)  . 


• wakes 


appearance  and  definition  should 
be  correlated  with  ship  actual 
speeds 


Note:  The  design  option  to  provide  wakes  on  the  radar  display 

was  not  selected. 


3.  PPI  DISPLAY  CHARACTERISTICS  (fidelity  dimensions  ©©> 
. range  strobe 

„ sweep  line 

• offset  features 

• target  designation  ^feature 

4.  SELF  TEST  INDICATIONS  (fidelity  dimensions  ©© ) 

5.  VISUAL  DISPLAY  REQUIREMENTS  (fidelity  dimension  (T)  ) 

« acuity  ; i-/'; 

• luminance 
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# refresh  rate 

• contrast  sensitivity 
» resolution 


6.  DISPLAY  REQUIREMENTS  FOR 

VECTORED  ATTACKS  (fidelity  dimensions  Kl)(£)  ) 

Symbology  is  required  for  WSA-1  (symbol  generator)  to  show:  water 

entry  point  for  aircraft  launched  torpedo;  contact  range  and  bearing; 
and  position  of  DASH  vehicle . 


Note:  Current  WSA-1  symbology  in  the  14A2  utilizes  the 

following  designs.  These  are  adequate. 


4-  indicates  water  entry  point  as  determined 

by  weapon  motion  analysis  in  UB/Plot 

0 Indicates  sonar  contact  range  and  bearing 

H Indicates  range  and  bearing  input  from  fire 

control  radar  (DASH  position) 


Note:  The  radar  in  the  14A2  has  no  sweep  line  and  there  is  no 

offset  feature.  Surface  ships  are  presented  as  blips  that  are 
not  proportional  to  the  length,  aspect  angle  and  range  of  the 
ship.  Surfaced  submarine  blips  are  also  consistent  in  size 
regardless  of  aspect  and  the  intensity  of  presentation.  Periscope 
depth  submarines  have  fixed  length  blips  with  an  intensity  based 
on  sea  state;  to  increase  realism,  presentation  of  the  blip  is 
interrupted  at  random.  Aircraft  have  fixed  length,  fixed  intensity 
blips  within  a representative  maximum  detection  range.  No  sea 
return  is  provided  on  the  radar  display.  It  appears  that  high 
fidelity  representation  was  not  selected  due  to  the  prohibitive 
costs  (e.g.,  sweep  servo  generator)  when  compared  to  the 
additional  training  value  achieved. 


VEHICLE  DYNAMICS 

1.  VEHICLE  CHARACTERISTICS  (fidelity  dimensions  ©CD©  > 

Own-Ship  - Based  on  the  types  of  surface  ships  that  will  send 
“ ' teams  to  the  14A2  device;  own-ship  dynamics  should 

be  a composite  of  FRAM  I conversions,.  1040/1052 
j DE,  DEG,  DLG,  and  C LG . 


523 


NAVTRADEVCEN  69-C-0298-1 


Note:  Present  vehicle  dynamics  in  the  14A2  are  a 

composite  of  a DD  type  (Gearing  Class)  and  a CLG. 

Support  Destroyers  - Two  assist  ships  are  required  to  provide 
the  training  capability  defined  for  SAU  operations  in 
the  training  objectives.  The  vehicular  dynamics  are 
those  available  for  own- ship. 

Note:  The  14A2  employs  the  composite  dynamics  as 

defined  for  own- ship. 

Aircraft  - Three  aircraft  are  required  to  provide  the  VECTAC 
operations  as  defined  in  the  training  objectives  for 
SAU  missions.  The  characteristics  should  represent 
VS,  VP  and  HS  dynamics  in  any  combination  of  the 
three,  6r  when  DASH  operations  are  involved,  any 
two  aircraft  combinations  plus  one  DASK  are  employed. 
DASH  Characteristics  are  those  of  the  actual  vehicle. 

Aircraft  Carrier  - CVS/CVA  Vehicle  Characteristics  are  desired 
to  provide  more  realistic  radar  and  sonar  returns  . 

Also  carriers  are  employed  in  screening  operations 
(i.e.  , ASW  in  the  vicinity  of  the  screen). 

Note:  The  second  target  submarine  in  the  14A2  (sub  2)  is  used 

to  represent  the  carrier.  Since  vehicle  size  differentiation  is 
not  portrayed  in  the  device,  the  use  of  the  second  sub  for  the 
carrier  is  not  a drawback  for  training  as  currently  accomplished. 

The  vehicle  dynamics  appropriate  to  the  types  specified  above 
should  be  programed  to  include: 

Speed  range 

Acceleration/ deceleration  characteristics 

Ordered  course 

Turn  rate 

Turning  radius 

Dive /climb  rate 

Ordered  depth/ altitude 

Response  lag 

The  programed  vehicular  dynamics  should  result  in  the  appropriate 
relative  motion  of  vehicles  displayed  in  sonar,  radar,  NC-2/DRT  and  appro- 
priate program  envelopes  of  speed,  sea  state,  and  water  depth  should  result 
in  the  generation  of  non-submarine  contacts  on  sonar  (ex.  , surface  ship 
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wakes  and  knuckles),  radar  (wakes),  and  also,  submarine  contacts  such 
as  hydrophone  effects,  wakes  and  knuckles. 

Note:  These  characteristics  are  not  provided  in 

Device  14AZ  . 


Own-ship  dynamics  should  also  be  represented  in  the  functioning 
of  ship  control  instruments  (e.g.  , gyro-compass,  pit  log,  rudder  angle 
indicator,  and  Mk  78  position  indicator). 


D.  WEAPONS  SUBSYSTEMS 


1.  OWN- SHIP 

ASROC 

Battery 

ASROC 
Mis  sile 


(fidelity  dimensions  ©(D ) 
Capable  of  adjustment  in  train  and  elevation  angle 


Trajectory  program  to  cover  present  firing 
range  capabilities 


MkC'X")  Nuclear  Depth  Charge 


Mk  25 

AWTT  - Fixed  train  angle 

Mk32 

AWTT  - Trainable 

Note:  The  Mk  32  ^s  not  trainable  in  the  14A2;  it  is  fixed  at  45°. 


ASW 

Torpedoes-  Current  torpedo  dynamics  for  Mk  44/1  and 
Mk  46/0  should  be  simulated  for  ASROC, 
AWTT,  and  DASH  delivery,  to  include  run 
out  patterns  and  homing  features  . 


2.  SUPPORT  UNITS 

VS,  VP,  HS 
and  Surface 
Support 

Units  - Capable  of  firing  the  above  ASW  torpedoes. 

Note:  In  14A2,  only  the  support  aircraft  can 

release  weagpns-.  An  additional  training  objective 
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could  be  achieved  in  coordinated  SAU  attack 
if  support  units  had  a weapon  firing  capability. 
Also,  friendly  unit  hydrophone  affects  are 
desired  but  not  available. 

3.  TARGET  SUBMARINES 

Torpedoes  - Capable  of  firing  torpedoes  against  own- ship 
or  surface  support  units. 

Note:  14A2  submarine  can  fire  only  one  torpedo 

and  only  on  own- ship.  No  display  of  weapons 
actions  (e.g.  , water  splash  point  of  air  to  water 
missile)  is  provided.  Tactical  training  capability 
can  be  extended  by  enabling  the  submarine  to 
fire  salvos  of  three  torpedoes. 

4.  MATERIAL  COUNTERMEASURES 

Fanfare  - Towed  noisemaker  should  be  simulated  in  terms 

of  streaming  and  activation.  It  should  be  pro- 
gramed for  own- ship  and  available  for  other 
surface  support  units. 

Note:  In  the  14A2,  Fanfare  is  available  only  for 

own-ship.  If  available  for  other  surface  units, 
it  would  provide  another  source  of  degradation 
for  the  Sonar  display. 

E.  MALFUNCTION  CAPABILITIES  (fidelity  dimensions  ) 

A design  option  for  the  device  involves  incorporating  types  of  malfunc- 
tions as  a means  of  exposing  teams  to  various  real-world  contingencies  and 
as  a means  for  increasing  the  difficulty  within  groups  of  training  exercises. 

Note:  14A2  design  provides  a malfunction  capability  only  for 

weapons  in  terms  of  launch  functions.  Weapon  selection  on 
the  Attack  Director  Firing  panel  can  indicate  failure  in  cells, 
train  and  elevation  angle,  or  indicate  a dud.  No  other  classes 
of  malfunction  are  available. 

Additional  malfunction  capabilities  important  to  satisfying  the  training  objec- 
tives could  include  the  following: 

Sonar  - audio,  video  failures 

§26A: 
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Complete  failure  would  force  own-ship  to 
execute  a Deep  Creep  attack  (own-ship 
Vectored  in  to  attack)  • 


NC-2 

Plotter  - Compass  rose  failure  would  force  the 

utilization  of  the  Halifax  plotting  method. 

Green/ red  bug  failure  would  force  the 
use  of  manual  plotting  procedures. 

Radar  - Degrading  of  resolution  (e.g.  , range,  video). 

Intermittent  failure  would  force  the  use  of 
ASAC  dead  reckoning  procedures. 


Attack 

Director 

(attack 

plotter 

section)  - Cursor  or  flasher  light  failure  would  force 

a fire  control  solution  via  CIC  . 

Weapons  - Propulsion  failure  would  force  the  team  to 

initiate  quick  reattack  procedures. 


RT  Nets  and 

MC  Circuits  - Communications  failures  would  force  a 
redesignation  of  RT  Nets  and  the  use  of 
the  MC  system. 

F.  FIRE  CONTROL  DISPLAY  (fidelity  dimensions  CDCD  ) 

REQUIREMENTS 


• sensor  data  handover  (target  range  and  bearing  information) 

• range  and  bearing  indication  of  target  (target  position)  (flasher 
light) 

• own  vehicle  display  symbol  (on  geographic  display) 

• cursor  dynamics  and  control  (cursor  display  for  solution  of 
target  course;  crank  control;  speed  shadow  control  for  solution 
of  target  speed) 
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• status  information 

channel  assignments  and  loading 
weapon  status 

rail  status  (up/ down,  loaded/ empty) 

G.  DELIBERATE  DEPARTURES  FROM  REALISM 


This  deals  with  special  features  of  the  14A2  device  where  deliberate 
deviations  in  design  from  the  operational  equipments,  other  than  the  engin- 
eering fidelity  continuum,  are  installed  to  enhance  training.  Prime  examples 
of  this  are  the  use  of  target  enhancement  techniques  on  trainee  displays,  or 
providing  immediate  knowledge  of  the  results  cf  performance  directly  on  a 
display  in  a trainee  compartment. 

No  real  use  is  made  of  this  design  option  in  device  14A2  with  the 
exception  of  the  Bridge  compartment  and  Radar.  The  helm  employs  rotary 
controlled  potentiometers  for  setting  own-ship  course  and  speed  instead  of 
actual  ship  controls;  the  functional  capability  and  the  PPI  display  of  the 
radar  are  different  from  that  found  aboard  ship.  The  radar,  however,  may 
be  described  as  lacking  in  engineering  fidelity  more  so  than  as  a deliberate 
departure  in  design  from  operational  realism. 

II.  TARGET 

A.  SONAR 

1.  TARGET  CHARACTERISTICS  (fidelity  dimensions  O©  i 

. Number  of  targets  to  represent  at  a given  time  in  the  mission 
environment  (one  submarine  is  displayed  in  the  14A2). 

. Classes  of  targets.  Both  nuclear  and  conventional  submarine 
vehicle  dynamics  are  required,  equivalent  of  Russian 
‘’November"  and  "Foxtrot"  vehicles. 

Note:  The  device  employs  the  dynamics  of  the  Nautilus 

class  (nuclear)  and  the  Guppy  II  class  (conventional). 

Two  target  submarines  are  programed  in  the  device 
although  only  one  submarine  is  usually  represented  in 
any  engagement. 

Note:  Accelerations  and  decelerations  for  nuclear  sub- 

marines are  not  realistic  in  the  14A2  . For  example,  to 
decelerate,  one  must  freeze  the  movement  and  knock 
down  in  5-knot  units  to  achieve  the  desired  speed. 
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. Target  appearance  / disappearance/ attenuation  as  a function 
of  range  and  sea  conditions.  The  detection  range  available 
should  correspond  to  the  full  range  achieved  operationally. 

Note:  A serious  limitation  in  present  design  of  14A2  is 

that  the  maximum  detection  ranges  available  are  not 
achieved  in  the  device  . An  empirical  study  by  Dunlap 
and  Associates,  Inc.  , utilizing  device  14A2A  indicated 
detection  ranges  under  10,000  yards.  For  example,  a 
detection  range  of  only  9,000  yards  was  achieved  on  the 
device  for  these  conditions:  sea  state  1,  100  foot  layer, 

submarine  depth  of  50  feet  with  a beam  aspect.  This 
shortened  detection  range,  hampers  the  training  capability 
in  that  it  prevents  extended  utilization  of  standoff  weapons 
systems.  Full  training  potential  with  VECTACS  and  DASH 
are  not  exercised  since  the  greatest  difficulty  in  control 
occurs  at  the  greater  distances  resulting  from  greater 
range  and  bearing  errors. 

. Target  movement  as  a function  of  range,  bearing  and  aspect 
changes . 

2.  TARGET  VEHICLE  RETURNS  (fidelity  dimensions  (T)(T)  ) 

video  - appropriate  blip  size, 
shape  and  brightness  persist- 
ence for  brightness  modes 

audio  - doppler  effects  (up, 
down,  no)  via  increase/no 
increase/ decrease  in  pitch 
over  transmission  frequency 


. aspect  (bow,  beam, 
stern) 


Note:  There  is  no  video  correlate  for  aspect  in  the  14A2. 


. cavitation  effects 

• hydrophone  effects 
from  torpedo 


video  - appropriate  noise 
spoke  for  speed,  depth,  range 
and  BT  envelopes 

audio  - appropriate  audio  char 
acteristics  for  speed,  depth, 
range  and  BT  envelopes 


. knuckles 
# wakes 


video  - appropriate  blip  size, 
shape,  and  brightness  per- 
sistence for  brightness  modes 
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audio  - appropriate  audio 
characteristic  s 


Note:  The  design  option  for  providing  false  targets  is  not 

employed  in  the  14A2  . Also,  submarine  wakes  are  not  pres- 
ently provided.  With  a display  of  wakes  the  situation  exists 
for  the  sonar  operator  to  erroneously  fftrack  offM  on  wake 
trails.  It  also  provides  an  evasive  capability  for  the  sub- 
marine for  exercises  emphasizing  tactical  deception  and 
greater  own- ship  threat  evaluation. 

i.  RADAR 

1.  TARGET  CHARACTERISTICS  (fidelity  dimensions  ©©  ) 

• Total  number  of  targets  to  represent  in  mission  environment 

• Classes  of  targets 

• Number  of  targets  displayed  simultaneously/ instantaneously 
(number  that  the  trainee  is  -apable  of  processing  in  his  span 
of  control) 

. Target  appearance/disappearance/attenuation  as  a function 
of  range /altitude  and  environmental  conditions  ( e . g . , radar 
fades,  height  of  antenna) 


• Detection  ranges  should  correspond  to  full  range  achieved 
operationally 


. Capability  for  add-on  of  new  targets  (for  updating) 

2.  TARGET  RETURNS  (fidelity  dimensions  ©®  > 

• submarine 


surfaced 

periscope  depth/ 
snorkel 


blip  proportional  in  size  , shape 
and  brightness  to  vehicle  type, 
aspect,  range  and  range  scale 
selected 


Note:  Both  of  these  factors  are  provided  in  the  device,  but  the 

intermittency  program  affects  the  periscope  returns.  Blip  size 
and  shape  for  surfaced  submarine  is  not  available. 
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feathers 


appearance  and  intensity 
correlated  with  submarine 
speed  at  periscope  depth 


Note:  The  display  of  feathers  is  not  provided  for  in  device  14A2. 


« aircraft 

speed  and 
maneuvers 

relative  motion 

lock-on 

indications 

target  identification 

( e . g . , SIF) 

* surface  vehicle 

speed  and 
maneuvers 

3.  TARGET  DISTORTIONS 

• garbling  (signal  overlap) 


blip  proportional  in  size,  shape 
and  brightness  to  vehicle  type, 
aspect,  range  and  range  scale 
selected 


blip  proportional  in  size,  shape 
and  brightness  to  vehicle  type, 
aspect,  range  and  range  scale 
selected 

(fidelity  dimensions  ©©  > 


III.  MEDIA 


A.  SONAR 

1.  ENVIRONMENT 

• Wind  direction  and  speed 


(fidelity  dimensions  ) 


0°  - 360°/0- 100  knots 

The  design  requirement  is  the  capability  of  wind  direction  and 
speed  to  affect  all  air  operations  and  to  affect  sea  state. 
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Note:  14A2  design  employs  wind  direction  and  speed 

to  affect  only  DASH  and  ASROC  (automatically). 

Aircraft  operations  are  not  affected  by  wind  forces. 

. Sea  state 

Sea  state  range  of  1 to  6,  correlated  with  wind  speed  to  cause 
degradation  on  sonar  (reverberations)  and  on  radar  (sea  returns). 

Note:  14A2  design  does  not  correlate  sea  state  with  wind 

speed.  Sea  states  of  1 to  6 are  achieved  with  a reverberation 
control  for  sonar. 

# Bottom  depth/ type 

The  desired  option  is  to  provide  a controllable  depth  from  100 
feet  to  2000  feet  and  showing  characteris tic s of  rocky  and  sandy  bottoms  for 
a realistic  sonar  environment*  This  will  also  permit  training  in  shallow 
water  operations  . 

Note:  14A2  design  does  not  have  controllable  bottom  depth.  A 

2000  foot  bottom  is  assumed  in  the  program.  The  option  not  to 
depict  bottom  characteristics  is  presumably  based  on  the  decision 
to  omit  the  training  objective  of  shallow  water  operations. 

# Thermal  gradient 

Representative  sampling  of  the  layer  depths,  operationally 
experienced,  are  required. 

[Present  design  employs  the  following  increments: 

0,  50,  100,  200,  300  and  isothermal. 


„ Land  masses 


A desirable  design  option  is  to  provide  sonar  and  radar  depictions 
of  two  representative  harbor  areas  for  team  training  in  shallow  water  opera- 
tions. This  is  not  simulated  in  the  14A2  presumably  based  on  the  decision 
to  omit  the  training  objective  of  shallow  water  operations. 


# Ocean  area 


X-Y  coordinate  system  (flat  plane  of  reference)  encompassing 
a maximum  of  64,000  x 64,000  yards  area  is  sufficient  to  provide  the  tactical 
environment  specified  by  the  training  objectives. 
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o Altitude 

A ' Z1  coordinate  representative  of  sea  level  to  5,000  feet  for 
fixed-wing  aircraft  and  sea  level  to  3,000  feet  for  helicopters  is  sufficient 
for  air  operations. 

2.  RANGE  (fidelity  dimension  (T)  ) 

. envelop  of  detection  ranges  that  are  achieved  operationally 
(appearance/attenuation  of  targets) 

3.  NON  TARGET  RETURNS  (fidelity  dimensions  ©@© ) 

a)  Non-Submarine  Returns  - Natural 

. Reverberations  I Appropriate  video  and  audio 

I characteristics 


O 
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Note:  Reverberation  simulation  is  not  satisfactory  in 

the  14A2;  it  is  presented  via  a random  generator,  with 
a control  labeled  11  SEA  STATE”  (1  through  6).  The 
sea  state  is  not  correlated  with  wind  speed.  Thus  , un- 
realistic events  may  be  achieved  (e  . g.  , sea  state  1, 
windspeed  22  knots). 


Bottom  returns 
(sandy,  rocky) 


Video  characteristics  for  typical 
”soft,f  and  "hard11  bottom  con- 
figurations 

Appropriate  audio  characteristics 


Note:  Absence  of  bottom  return  in  the  14A2 

limits  training  in  shallow  water  exercises. 


Fish,  whales,  reefs.  Appropriate  blip  size,  shape  and 

wrecks  and  pinnacles  brightness  for  brightness  modes 


Appropriate  doppler  effects, 
echo  quality 

Note:  False  target  options  are  not  provided  in  14A2 

s imulation . This  obviates  the  necessity  for  the  use 
of  bottom  charts  and  CIC  keeping  track  of  underwater 
obstructions  . 
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b)  Non-  Submarine  Returns  - Submarine  Launched 

Appropriate  audio  characteristics 
Appropriate  noise  spoke 


Torpedo  hydrophone 
effects 


False  target  cannisters, 
decoys,  jammers 
(NEA  beacon) 


Appropriate  video  characteristics 
Appropriate  audio  characteristics 


Note:  False  targets  not  provided  for  in  device  design 

(see  above).  Submarine  tactical  countermeasures  are 
not  employed. 

c)  Non -Submarine  Returns  - Own- ship  Launched 

Appropriate  video  characteristics 


Torpedo  hydrophone 
effects 

Fanfare 


Appropriate  audio  characteristics 


4. 


Note:  No  torpedo  hydrophone  effects  are  displayed  on 

Sonar  resulting  from  own-ship  fired  torpedoes.  Fanfare 
is  not  displayed  on  Sonar. 

BACKGROUND  CLUTTER/NOISE  (fidelity  dimensions  (T)  (j)  ) 
ON  DISPLAY 


Summary:  Sonar  Video  Simulation 


The  Sonar  video  simulation  is  cleaner  in  what  is  portrayed  than  in  the 
operational  environment  and  it  is  also  less  flexible.  There  is  a limited 
capability  for  deliberately  degrading  equipment  performance.  The  Sonar 
operator  is  denied  the  clas sification  task  since  most  of  the  parameters 
associated  with  uncertainty  in  target  identification  are  not  represented  in 
the  device.  With  the  classification  function  present,  more  effective  use 
of  the  device  could  be  made  in  a program  of  training  where  additional 
training  value  would  accrue  in  complex  multi-unit  exercises; involving  tactical 
employment  of  own-ship  (i.e.  , fighting  the  ship). 

B . RADAR 

1.  ENVIRONMENT  (fidelity  dimensions  ® ©) 


wind  direction/ speed 
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• sea  state 


. altitude 


• land  mass/ elevations 


2.  RANGE 


(fidelity  dimension  © ) 


• envelop  of  detection  ranges  achieved  operationally 
(appearance /attenuation  of  targets) 


3.  NON- SUBMARINE  RETURNS 


* second  sweep  echoes 


blip  characteristics  should  be 
similar  to  operational 
characteristic  s 


• radar  decoys 


Note:  These  are  not  available  in  the  device.  This  design 

option  would  provide  sources  of  false  contact  information. 


Note:  Sea  return  is  not  available  in  the  device.  This  design 

option  would  increase  the  difficulty  in  radar  detection. 

• land  mass  characteristics 

Note:  Land  masses  are  not  displayed.  This  design  option 

would  increase  the  fidelity  of  training  in  shallow  water 
operations  . 


sea  return 


density  and  brightness  should 
be  correlated  with  sea  state 
and  range 


4.  BACKGROUND  CLUTTER/NOISE  (fidelity  dimensions 
ON  DISPLAY 


5,  INTERFERENCE  EFFECTS  (fidelity  dimensions  ©©  > 

FROM  MULTI- SHIP 
ENVIRONMENTS 


? ■ 
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4.5.  1.2.3  Simulation  Elements  Summary.  The  simulation  elements 

involved  in  installing  the  tactical  environment  for  training  must  be  pertinent 
to,  and  of  sufficient  number  and  range  of  values  to  provide  the  means  for 
accomplishing  the  classes  of  training  exercises  described  earlier.  The 
total  array  of  the  simulation  elements  presently  organized  in  the  device, 
provides  for  effective  ASAV  team  training.  That  is  to  say,  with  the  device 
configuration  as  a given,  effectively  developed  utilization  procedures  (e.g.  , 
a graduated  series  of  training  exercises  such  as  indicated  in  the  section 
on  training  objectives)  can  accomplish  training  from  Fire  Control  Solution 
and  Launch  Procedures  in  a one-on-one  situation  through  a range  of  SAL 
exercises  (the  SAU  context  places  the  greatest  design  requirement's  on  the 
device).  For  example,  a number  of  complex  SAU  missions  can  be  accom- 
plished in  Device  14A2 . Figure  51  shows  the  SAU  missions  that  can  be 
accommodated.  An  "X"  denotes  a primary  training  mode;  a nB"  denotes 
a secondary  or  backup  mode;  a ’’P"  denotes  a possible  training  mode. 

Blank  cells  denote  either  non-relevancy  or  no  training  possibility. 

However,  a number  of  the  design  options  indicated  earlier 
and  not  selected  for  the  14A2  have  effects  on  training.  Some  are  easily 
defined,  others  are  not,  but  all  are  of  importance  in  human  factors  design. 
Some  examples  of  these  effects  are  shown  below. 

4.5.  1.2.3.  1 Signal  Characteristics  — the  design  goal  is  to  provide 

target  signals  which  are  realistic  of  the  range  of  real-world  events.  The 
14A2  is  not  a large  multi-signal  environment,  most  often,  only  a single 
submarine  signal  is  displayed  plus  a maximum  of  three  aircraft  and  two 
assist  ships  and  the  possible  use  of  the  second  submarine  (SU-2)  as  a 
vehicle.  The  characteristics  of  the  target  must  be  well  defined.  For  the 
most  part,  Sonar  simulation  is  adequate,  yet  complete  fidelity  is  not 
achieved,  for  example,  provision  of  classification  cues.  The  effect  of 
this  is  to  introduce  a lack  of  realism  in  exercise  structure.  There  is  less 
than  desired  involvement  of  the  Sonar  and  the  CIC  subteams  in  tactical 
decision  making  sequences.  Certain  training  objectives  are  not  achievable 
and  there  are  certain  losses  in  controlling  the  exercise  difficulty.  Yet 
the  effects  of  this  on  transfer  of  training  are  not  clear  except  on  logical 
grounds  which  point  clearly  to  the  desirability  or  training  with  the  classi- 
fication task  in  ASW.  But  if  teams  are  exposed  minimally  to  training  (such 
as  a recurring  schedule  of  two  days  of  training  in  a three— month  period), 
the  cost  factor  outweighs  the  training  benefit,  since  much  of  the  team 
efforts  are  devoted  simply  to  refining  their  team  procedural  skills. 

Usually,  teams  that  return  to  the  trainer  in  a six-month  period  show 
better  than  a 50  percent  turnover  in  personnel. 

4.5.  1.2. 3.2  Absence  of  Simulation  Elements.  A number  of  the 

design  options  that  would  enhance  the  training  capability  but  are  not  reflected 
in  present  14A2  design  (Sonar  and  Radar  Display  limitations , absence  of 
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certain  environmental  features,  malfunction  capabilities),  serve  to  increase 
training  value  in  that  they  encourage  greater  involvement  in  task  performance, 
increase  problem  difficulty  levels  and  confound  tactical  solutions  thereby 
heightening  the  tactical  decision-making  requirements.  Yet,  a program  has 
not  been  undertaken  to  obtain  evidence  that  these  will  produce  a greater  trans- 
fer of  training  to  the  real-world.  At  best,  it  can  only  be  logically  or  intui- 
tively stated  based  on  the  research  literature,  that  training  -vill  be  enhanced. 
Although  training  value  is  unknown,  it  is  clear  that  greater  raining  flexibility 
is  achieved  and  this  in  itself  is  desirable.  What  must  be  done  is  to  weigh 
these  logical  gains  against  training  purpose  and  make  the  necessary  cost 
tradeoffs  (which  at  present  is  a highly  imprecise  art). 

So  far  as  the  unretolvable  areas  of  design  are  of  con- 
cern, the  question  is,  how  far  can  we  bacK.  off  from  100%  fidelity  and  still 
achieve  our  training  purpose?  Since  training  value  data  are  lacking,  the 
question  of  "nice  to  have"  vs.  "needed"  in  simulation  is  inexactly  resolved. 
Quite  often,  as  history  has  shown,  solutions  are  based  on  "unusual"  criteria. 
Related  to  this  is  the  question  of  how  far  must  we  go  in  design  to  get  the 
desired  performance?  Again,  the  question  is  tied  in  with  training  purpose 
and  the  objectives  of  training,  but  cost  is  an  overriding  feature.  Unfortunately, 
more  research  issues  than  facts  have  been  generated  by  these  crucial 
questions  . 

4.5.  1.3  Output . Identification  is  made  of  the  crucial  factors  in  repre- 

senting the  operational  environment  in  the  trainee  compartments  . The 
functional  characteristics  of  trainee  compartment  design  consider  most 
prominently  the  issues  involved  in  achieving  simulation  fidelity  in  creating 
an  environment  for  learning.  The  completion  of  this  phase  yields  the 
following: 


Configuration  of  the  synthetic  system  (in  terms  of  the 
extent  of  representation,  the  hardware  components  in 
the  trainee  compartments  and  specific  equipment 
requirements  . 

Definition  of  operational  task  structure. 

Definition  of  the  simulation  elements  (parameters  to  be 
simulated  which  govern  the  range  and  complexity  of  task 
representation  in  the  device  and  which  are  required  to 
achieve  training  purpose  and  the  training  objectives). 

Special  features  in  trainee  station  design  which  presumably 
will  enhance  training  value  in  terms  of  greater  transfer 
of  training  to  the  operational  system. 
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4.6  THE  MANAGEMENT  OF  TRAINING- -STRUCTURING, 
CONTROLLING  AND  MONITORING  TRAINING  AT  THE  INSTRUCTOR 
STATION.  In  addition  to  the  design  issues  involved  in  installing  the 
environment  in  which  learning  can  take  place  (simulation  of  the  trainee 
compartments),  there  are  design  issues  associated  with  the  structuring, 
control  and  monitoring  of  training.  This  involves  the  management  of 
training  at  the  instructor  station,  i.e.  , how  the  device  will  be  utilized 
as  a training  tool. 


The  human  factors  design  of  tlie  instructor  station  complex  emphasizes 
the  idea  of  design  for  training  utilization.  Given  the  environment  for  learning 
(trainee  compartment  design),  the  concern  is  for  how  the  synthetic  environ- 
ment will  be  used  for  training  and  for  creating  and  monitoring  training  situa- 
tions in  order  to  shape  team  behaviors  and  to  optimize  the  training  strategies 
for  teams  during  the  course  of  instruction.  The  design  requirement  is  to 
provide  a capability  for  structuring  training  so  that  the  important  mission 
and  system  events  can  be  installed  and  controls  provided  to  insure  that  these 
events  occur  in  prescribed  ways  at  prescribed  times.  This  calls  for  a flex- 
ibility in  training  capability  so  that  the  defined  situations  can  occur  at  the 
desired  times  in  order  to  satisfy  the  training  objectives. 

A tuman  factors  design  pathway  is  developed  to  provide  an  instructional 
capability  consistent  with  the  training  purpose  and  the  design  of  the  trainee 
compartments.  The  approach  consists  of  evaluating  the  information  require- 
ments for  the  instructor  station  and  examining  the  design  alternatives  and 
selecting  those  design  options  most  relevant  to  achieving  the  desired  training. 
Defining  the  design  pathway  involves  evaluation  within  the  following  areas: 


a.  The  training  device  mode  of  operation 

b.  Analysis  of  instructor  functions 

c.  Instructor  requirements 

d.  Display  requirements 

e.  Control  requirements 

f.  Monitoring  and  control  of  training  (enroute  during  the  exercise) 

g.  Pre-mission  requirements 

h.  Post-mission  requirements 

i.  Measurement  system  requirements 
j . Communications  requirements 

k.  Overall  layout  and  workplace  requirements 


4.6.1  Mode  of  Operating  the  14A2  Device 

The  initial  consideration  involves  specifying  the  operating  modes 
at  the  instructor  console  in  controlling,  monitoring  and  evaluating  team 
training.  These  modes  range  from  manual  operation  to  semi-automated 
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and  automated  operation.  The  selection  of  operating  mode  must  be  considered 
carefully  because  of  the  hardware  and  computer  programing  requirements. 

Note:  The  decision  was  made  to  operate  the  instructor 

station  in  the  manual  mode.  Our  analysis  will  examine 
the  requirements  for  achieving  this  capability  (a  dis- 
cussion of  instructor  station  modes  is  provided  in  Paragraph 
2.8.1  of  Section  II  of  this  report). 

Decisions  must  be  made  about  instructor  involvement  in  the  total 
training  mission  sequence  beginning  with  pre-exercise  briefing  and  the  setting 
up  of  initial  conditions  for  an  exercise,  the  control  of  mission  events  via  a 
script  of  event  setups  during  the  actual  running  of  the  mission  scenario,  and 
ending  in  post- exercis e critique. 

4.6.2  Analysis  of  Instructor  Functions 

An  identification  and  analysis  of  instructor  functions  in  the  manual 
mode  is  undertaken  for  the  activities  involved  in  pre-mission  setup,  enroute 
mission  operations  and  post-mission  activities.  The  display  control  and 
communications  requirements  are  specified  for  the  following  classes  of 
operations  and  instructional  functions  . 

* initialize  system  for  mission  operation 


O 

ERIC 


checkout  of  subsystems 
power 

projection  display  equipment  (Universal  Program  Generator) 
setup  of  initial  conditions 

student  briefing  (including  demonstration  mode  where  applicable) 

monitor  and  control  of  team  performance  (hardware  provisions 
for  the  following): 

exercise  start  (COMEX) 
exercise  freeze 

override  of  scenario  (as  applicable) 
demonstration  mode 

control  of  events  throughout  mission  (time  and  event 
control  of  problem  manipulatible s via  graduated  series 
of  training  exercises  in  the  utilization  guide) 

problem  speed  control  (fast  time  (2X)  to  reduce  runout 
time  required  when  a team  makes  a wrong  weapon 
as  signment) 
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communications  with  trainees,  and  remote 
instructors 

instructional  assists  for  unburdening  the  instructor 
(e  * g.  , hit  assessment  light  (computer  function) 

. evaluation  and  scoring  requirements 

. mission  termination 

. post-mission  critique 

equipments  for  critique  (projection  display  equipment, 
audio  recording) 

The  display , control  and  communications  requirements  for  the 
identified  instructor  functions  can  now  be  analyzed  in  the  detail  necessary 
to  achieve  the  desired  design  pathwa.y  for  the  instructor  station.  Table  40 
presents  the  instructor  functions  ana. lysis. 

4.6.3  Instructional  Staff 

Based  on  workload  estimates,  instructor  manning  requirements 
range  from  three  to  eight  according  to  type  of  training  exercise  being  con- 
ducted. Early  in  the  training  sequence,  a minimum  of  three  instructors 
appear  to  be  needed  in  certain  exercises  to  achieve  the  balance  between 
control  of  the  exercise  and  assisting/ observing  team  performance  in  the 
compartments.  For  example,  an  ASROC  RTDC  attack  exercise  requires: 

One  instructor  at  the  master  console  to  conduct  the  setup  of 
initial  conditions,  control  the  target  submarine  and  monitor  the 
1JS. 

One  instructor  in  the  CIO  compartment  to  observe  and  assist 
performance  and  complete  a team  performance  checklist 
(assessment). 

One  instructor  in  the  XJB/!Plot,  Sonar  compartment  to  observe 
and  assist  performance  and  complete  a team  performance 
checklist  (assessment). 

An  example  of  a more  complex  exercise,  and  one  that  imposes 
the  greatest  instructor  manning  requirements  is  an  AWTT  44-1  torpedo 
attack  from  a sector  screen.  The  instructor  requirements  here  are  as 

follows: 
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TABLE  40.  ANALYSIS  OF  INSTRUCTOR  FUNCTIONS 

FOR  DEVICE  14A2 


Function 

Display 

Requirements 

Control 

Requirements 

Equipment 

System 

initialization 

Proper  ASROC 
launcher  position 

r 

Program  insertion 

114  Fire  Control 
System 

System  data 
cleared  (all  zeros) 

Insert  parameters 
of  the  vehicle 

Computer  tape 

Project  in 
system 

Initiate  & align 
the  projection 
system  including 
new  slides 

Time  clock  at 
time  zero 

Power-on 

indication 

Power-on  controls 

(Note:  the  above 

are  functions  per- 
formed by  the 
training  device 
operator  (TD) 

Exercise  set- 
up (initial 
conditions) 

Unit  types  (own- 
ship,  support 
units,  submarine 
target) 

- x,y  positions 

- depth/  altitude 
of  vehicles 
own- ship 
initial  course 
and  speed 

Vehicle  program 
designator 

For  specific 
vehicle,  controls 
to  enter  these 
data 

Master  control 
console  keyboard 

'* 

Weapon  selection/ 
loadout 

Delivery  system 

Environmental 

conditions 

- sea  state 
layer  depth 

- wind  direction/ 

Environment 

console 
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TABLE  40.  ANALYSIS  OF  INSTRUCTOR  FUNCTIONS 
FOR  DEVICE  14A2  (Continued) 


Function 

Display 

Requirements 

C ontr  ol 
Requirements 

Equipment 

Sea  area  (64  x 64 
miles  , 32  x 32 
miles) 

Controls  to  enter 
these  data 

Master  console, 
projection  system 

Communications 
circuits  selected 

Controls  for 
positioning 
selected  circuits 

* 

Weapons  load 

Controls  for  in- 
stalling full 
weapon  load 

Universal 
Program 
Generator  setup 

Controls  for  setup 
of  Universal 
Program 
Generator  (for 
tape  replay) 

Team  briefing 

Projection 

equipment 

Enroute  mission 

Own-  ship 

Enabling 

Required: 

operations  (in-* 
puts  as  required) 

po  sition 

Contact  (range  &: 
bearing  for  sub- 
marine position 
required  for 
contact  insertion 
converted  from 

controls 

Master  control 
cons  ole 

Vehicle  control 
console 

Display  control 
console 

x,y,z 

coordinate  s) 

Fire  control 
console 

Fire  control 
solution  read- 
outs 

Seascape  PPI 
console 

? 

L-L 

Friendly  weapon 
readout  (water 
entry  point) 

Vehicle  data 
c onsole 

Sonar  and  PPI 
console 

543 
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TABLE  40. 


AN  LYSIS  OF  INSTRUCTOR  FUNCTIONS 
FOR  DEVICE  14A2  (Continued) 


j Function 

Display 

Requirements 

C ontrol 
Requirements 

Equipment 

Weapon  preset- 
ting information 
(ISD,  floor) 

Sonar  operating 
status  and  FPI 

Time  readout 
from  contact  to 
attack  (firing 
interval) 

Timer 

Submarine 

maneuvering 

Keyboard 
commands  for 
maneuvering 

- 

Maneuvering/ 
control  of 
support  units 
(surface  aircraft 
guidelines  from 
A SAC  if  under 
positive  control; 
if  advisory  con- 
trol, as  briefed) 

Weapons  selec- 
tion and  weapon 
active  indication 

Hit  assessment 

Control  to  clear 
hit  light 

Submarine  tor- 
pedo launching 
( hy  dr o phone 
effects) 

Control  sequence 

Support  unit  tor- 
pedo firing 
(aircraft) 

54 
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TABLE  40.  ANALYSIS  OF  INSTRUCTOR  FUNCTIONS 
FOR  DEVICE  14A2  (Continued) 
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TABLE  40. 


ANALYSIS  OF  INSTRUCTOR  FUNCTIONS 
FOR  DEVICE  14A2  (Continued) 
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One  instructor  at  the  master  console  to  conduct  the  setup 
of  initial  conditions,  and  control  the  target  submarine. 

One  instructor  at  the  master  console  to  control  destroyer 

1 (SUl),  perform  the  functions  of  screen  commander  and 
issue/monitor  communications  over  the  screen  common 
net. 

One  instructor  at  the  master  console  to  control  destroyer 

2 (SU2)  and  monitor  communications  over  the  Cl  net. 

One  instructor  at  the  master  console  to  cont  rol  a support 
vehicle  (an  oiler),  perform  functions  of  the  OTC  and  monitor 
the  PRITAC  net. 

One  instructor  in  CIC  to  monitor /as sis t performance  and 
complete  the  CIC  team  performance  checklist  (assessment). 

One  instructor  in  UB/Plot  and  Sonar  to  monitor / assist 
performance  and  complete  the  team  performance  checklists 
for  UB  and  Sonar  (assessment). 

One  instructor  on  the  Bridge  to  monitorVas sis t performance 
and  complete  the  team  performance  checklist  (assessment). 

One  instructor  having  overall  cognizance  of  the  training 
exercise  and  monitor  the  1JS  circuit. 


In  cases  requiring  the  maximum  number  of  instructors  , doubling 
up  of  functions  may  be  necessary.  Where  possible  the  staff  should  attempt 
to  double  up  on  vehicle  control  functions  so  that  the  trainee  compartments 
may  be  covered  by  instructors. 

4.6.4  Display  Requirements 


A key  requirement  in  instructor  station  design  is  the  efficient 
display  of  team  performance  information.  The  specification  of  display 
requirements  is  largely  dependent  on  the  information  requirements  in 
terms  of  information  content;  amount,  variety  , and  speed  of  information 
change;  and  complexity.  The  information  classes  to  be  displayed  account 
for  the  following: 


o Mission  areas- -the  mission/tactical  environment/ relative 
geometry  in  the  pertinent  display  modes  (32  x 32,  64  x 64, 
128  x 128,  256  x 256  mile  areas)  . 
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. target  data 

target  contact--(onset/attentuation/disappearance  of 
targets  as  a function  of  range  and  other  pertinent 
factors) 

tracks 

rangc/bcaring  information,  doppler,  aspect,  depth/ 
altitude 


• Ow:>«hip  and  support  units 

identity,  position  and  tracks,  movements 

• Indication  of  the  system  state/ simulator  mode  of  operation 
and  the  present  phase  or  point  in  the  mission 

, Status  of  events  (e.g.  , event  symbol  visual  display) 

• Performance  information  on  each  team  (error  indications) 

, Indication  of  the  team  control/ tactical  actions  and/or  conse- 
quences of  team  actions,  for  monitor  and  control  of  training 
(e.g„>  range  scale  settings,  sector  selection,  selection  of 
operating  modes  such  as  MAD,  deployment  of  sonobuoys) 

• Indication  of  team  detection/ identification  of  targets  once 
displayed  (event  and  time  of  event),  Mote;  an  indicator 
light  is  used  in  the  HA2  to  show  that  a contact  is  made, 

• Indication  of  instructor  actions  once  initiated/ controlled 
over  time 


• Information  for  manual  control  functions  (eg,,  position, 
heading,  altitude/ depth  and  speed  information  for  "flying" 
an  aircraft  or  controlling  a submarine  target) 

4,  it. 4, 1 Display  Techniques.  Decisions  on  primary  display  techniques 

(media  and  mec ham trat i«en»ier , control  and  evaluation  of  training 
are  based  on  a correlation  of  the  information  requirements  for  the  system 
and  the  characteristics  of  classes  of  display. 


Repeater  displays  (reproductions  of  trainee  displays) 
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Event  on/off  indications  (e.g.  , indicators  depicting 
an  out-of- tolerance  event,  a trainee  mode  selection, 
or  instructor  insertion  of  an  event  (control/ display 
integration)  such  as  a malfunction) 

plotting  boards  (mechanized) 

Computer  generated  displays  — synthesized  multi-format/ 
integrated  displays  (e.g.  , Cathode  Ray  Tube  display  of 
alpha-numeric/ pictorial- situational  information) 


Note:  All  of  the  above  techniques  are  used  in  ‘ t?#-  14A2  except 

computer-generated  displays  (CRTs). 


4.6.4.  1.  1 Criteria  for  Defining  Primary  Display  Requirements. 

Decisions  on  primary  display  techniques  and  display  modes/format  require- 
ments should  consider  the  following  criteria. 

• Information  handling  capability  (amount,  types,  rapidity 
of  change,  number  of  trainees  under  control) 

• Multi-format  requirements,  high  density  Information 
display,  information  integration  (alpha-numeric  and 
situational/graphic  display  requirements  involving 
discrete  event  and  overall  situation  displays;  continuous 
alpha-numeric  error  readouts) 

* Rapid  mode  changes  for  monitoring,  control  and  evalua- 
tion, requirements  for  monitor  and  control  of  more  than 
one  subteam  sequentially 

. Display  flexibility  (with  relevant  controls) 

- mis sion/ geographic  areas 

- scale  depict!  ms 

- information  classes 

- display  modes 

- coding  of  displayed  information 

# Accuracy,  reliability  and  registration  (information 
alignment)  of  the  visual  d.s plays 

„ Expansion  capability  (for  future  display  requirements) 

. Visual  factors 
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acuity 

luminance 

. efresh  rate/flicker 
contrast  sensitivity 
resolution 
clutter 

4. 6.4.2  Ancillary  j-napiay  Requirements.  Requirements  are  also 

specified  for  those  displays  not  primarily  involved  in  the  instructional 
process  but  which  are  necessary  to  training  system  operation  (e.g.  , 
mission  timers,  system  status  displays,  etc.) 

4.6.5  Instructor  Console  Controls 


4.6.5.  I Control  Requirements  in  the  Mission  Sequence.  To  achieve 

full  use  of  the  instructional  capability,  a number  of  classes  of  controls  are 
required  to  operate  the  training  device  and  to  c * hie  the  shaping  of  behavior 
in  the  trainee  compartments.  The  control  class  ^ include  the  following: 

. Device  operation- -start,  freexe,  reset,  override. 

, Means  for  selecting  displays  and  display  modes  (multi-format 
displays) . 

e Means  for  controlling  vehicles  (e.g.  . controls  for  '’flying*'  an 
aircraft,  steering  a target  submarine). 

• Means  for  positioning  units  in  the  mission,  both  initially  at 
problem  start  and  during  the  mission. 

• Means  for  direct  access  to  the  device  computation  system 
(manual  input  controls  (keyboard). 

• Environmental  controls. 

• Communications  controls. 

• Means  for  simulating  failures/emergencies. 

. Means  for  inserting  mission  events  during  an  exercise 

• Means  for  eliminating  defined  programmed  events  (for  example, 
elimination  of  slug,  sonar  contact,  radar  contact) 
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4.  6.  5.2  Selection  of  Controls.  Decisions  on  the  selection  of  controls  for 

operating  the  instructor  station  and  controlling  the  training  are  based  on  the 
following  criteria: 

• Display-control  compatibility  (compatibility  with  tabular 
and  graphic  formats). 

• The  amount  and  types  of  information  and  speed  require- 
ments in  information  reception  and  transmission. 

• Number  and  size  required  to  avoid  clutter,  ambiguity  and 
needless  redundancy  because  of  the  potential  for  error  in 
usage . 

• Ease  of  operation,  particularly  where  instructor  to  trainee 
ratios  are  high,. 

• Efficiency  of  panel  space  usage. 

Instructor  console  controls  can  be  grouped  according  to  the 
functions  performed  (obtained  from  instructor  functions  analysis)  which  in 
turn  affects  the  choices  in  control  selection. 

4. 6.  5. 2,1  Mission/Instructional  Controls- -(on-line) . These  are  the 

major  groups  of  controls  used  for  structuring  and  controlling  training  during 
the  exercise,  and  range  from  switches  through  control-display  indicators  to 
manual  input  devices.  In  multi-man  training  systems  where  fast  access  to 
much  information  m multi-formats  is  necessary,  the  selection  of  manual 
input  controls  with  compatible  displays  is  an  important  decision  for  training 
management.  The  alternatives  available  in  the  selection  of  controls  include 
the  following: 


• Switch/indicator  array  (engraved  legends) 

• Digit  keyset 

• Alpha-numeric  keyboard 

• Joystick  (cursor  symbol  positioner) 

• Trackball  (cursor  symbol  positioner) 

• x-y  multi-gain  push  buttons  (cursor  symbol 
positioners) 
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» light  pen  (or  light  gun) 

Note:  The  14A2  employs  switch  indicator  arrays 

and  digital  keyboards . 

4. 6. 5.2.2  Scenario  Modification- -(off-line)  . These  are  the  pertinent 

controls  for  the  efficient  setup  of  the  initial  conditions  for  an  exercise  and 
for  enroute  modifications  in  an  exercise. 

4. 6.  5.2. 3 Ancillary  and  Power  Supply.  These  are  the  controls 

associated  with  the  operation  of  the  training  device,  e.g.  , power  to  instructor 
console,  mission  initiation,  emergency  stop,  lamp  test,  etc. 

4.6.  5.2.4  Communications.  These  are  standard  controls , functional 

replicas  of  existing  equipment  for  two-way  communications  with  trainees; 
with  remote  instructors;  and  with  other  associated  personnel. 

4.6.6  Monitor  and  Control  of  Training  (During  the  Training  Exercise) 

To  achieve  the  training  purpose  and  the  training  objectives,  the 
characteristics  of  the  instructional  capability  must  be  specified.  It  is 
desirable  that  14A2  design  provide  the  capability  to  exploit  two  major  levels 
of  team  training  in  the  mission  (screen,  SAU)  contexts  and  controls  and 
displays  must  be  provided  at  the  instructor  console  so  that  event  positioning 
at  prescribed  times  in  the  mission  scenario  can  be  set  up.  These  two  levels 
of  training  emphasis  are  outlined  briefly  below. 

Level  1 emphasises  the  operation  of  the  ASW  platform  in  which  a 
series  of  "givens"  are  provided  to  the  team  for  implementation,  that  is,  the 
approach  to  the  scene-of-action  (direct,  offset,  intercept),  search  plans 
(area,  intercept,  lost  contact),  and  attack  plans  and  weapons  systems  are 
specified.  This  level  of  training  provides  a single  thread  through  an  exer- 
cise in  that  a number  of  decisions  which  have  instructional  expediency  are 
made  for  the  team  (who  customarily  resolve  these  decisions  themselves  in 
the  operational  world).  The  emphasi?  at  this  level  is  on  training  in  coordi- 
nated ASW  procedures , i.e.  , can  the  team  accomplish  the  activity  sequences 
in  ASW  operations?  The  mission  profile  for  a SAU  operation  can  be  dia- 
gramed as  in  Figure  52. 

Level  2 does  not  employ  "givens"  but  requires  the  team  to  select 
and  implement  a course  of  action  required  in  an  engagement.  This  is  the 
realistic  simulation  of  the  complete  engagement  where  a team  performs  as 
in  the  real  world.  This  level  emphasizes  the  tactical  employment  of  the 
ASW  platform  since  in  complex  operational  situations  several  equally  good 
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(Single 

pathway- 

through 

exercise) 


Figure  52.  Level  1 , SAU  Mission  Profile  for  Device  14A2 
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solutions  to  tactical  problems  exist  so  that  differences  between  teams  may 
become  evident  only  in  the  way  a team  copes  with  the  ASW  situation. 

What  this  recognizes  is  that  there  may  be  a number  of  equally 
good  solutions  to  a problem,  and  that  multiple  pathways  (decision  points) 
exist  for  each  exercise.  For  any  exercise  in  Level  2,  only  the  conditions 
at  COMEX,  the  datum  characteristics,  the  contact  source  and  quality,  and 
the  submarine  maneuvers  are  under  instructor  control.  The  remainder  is 
developed  and  implemented  by  the  team  as  prescribed  by  the  mission  re- 
quirements , Thus,  equally  good  “multiple  pathways'1  must  often  be  considered 
for  each  exercise  (one  of  which  is  the  “single  pathway11  in  the  Level  1 counter- 
part) . The  problems  to  be  solved  are  set  in  a “free  play11  context  but  with  the 
structure  and  control  necessary  to  insure  standardized  exercises  in  training. 
The  mission  profile  for  a SAU  operation  can  be  diagramed  as  in  Figure  53. 

The  displays  of  necessary  team  performance  information  and  the 
controls  and  the  communications  necessary  to  develop  a training  strategy 
for  the  team  are  specified.  The  options  provided  at  the  instructor,  console 
for  structuring  and  controlling  training  include  the  following: 

m Provide  monitoring  and  control  of  performance  via 
displays  at  the  console.  This  information  is  used  in 
judgments  for  continuing  the  mission  as  planned.  Verbal 
communication  capability  with  trainees  is  required  through- 
out the  training  exercise. 

, Provide  demonstration  capability  for  a team  before  the 
briefing  is  begun. 

* Provide  freeze  capability  of  an  exercise  for  demonstration, 
guidance  or  administrative  hold. 

. Provide  for  control  of  the  pace  of  the  training  (e.g.  , variations 
in  the  maneuvering  of  the  submarine) 

4.6.7  Pre-Mission  Requirements 

The  hardware  requirements  in  pre-mission  activities  involve  the 

following: 


* System  initialization  confidence  checks,  power  requirements, 
program  insertion 

. Setup  of  initial  conditions  for  an  exercise. 
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Figure  53.  Level  2>  SAU  Mission  Profile  for  Device  14A2 
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The  manipula tables  that  are  controllable  in  the  setup  of  the  initial 
conditions  are: 

, Own-ship 

course 

speed 

position  (XY) 
equipment  malfunction 


# Support  units 

range 

bearing 

course 

speed 

altitude  (air <.  raft) 

# Target 

position  (including  contact  (xy)  at  prescribed  time) 

range 

bearing 

course 

speed 

depth 

type  submarine  (via  speed) 

aspect 

doppler 

false  target  (decoy,  i.e.,  multiple  echoes) 

# Environment 

sea  state 
layer  depth 
wind 


* Weapons 


ASROC 

DASH 

AWTT 


nuclear  depth  charge 
a/ c torpedoes 
submarine  torpedoes 
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4.6.8  Post-Mission  Requirements 


The  hardware  implications  in  instructor  post-mission  activities 
involve  special  equipment  for  the  critique  of  team  performance.  Since  no 
automated  preprogramed  scenarios  are  employed  (i.e.,  no  requirement  for 
recycling  vs.  shutting  down  the  system)  and  no  hard  copy  printouts  of  team 
performance  are  required  (i.e.  , for  on-line  critique  information  and  for 
sc-hool  record-keeping),  the  requirement  considers  only  equipment  for 
mission  replay.  What  is  needed  is  a visual  display  projection  (UPG  system) 
and  audio  recording  (tape)  for  the  reconstruction  of  a mission  just  completed. 

4.6.9  Measurement  Capability 


Measurement  is  undertaken  to  enable  the  instructor  staff  to  assess 
the  progress  and  the  outcomes  of  team  training.  Its  purpose  is  to  provide 
objective  descriptions  of  team  performance  necessary  for  a meaningful 
critique  of  an  exercise  and  to  provide  a data  base  for  the  school  record- 
keeping function.  The  exercise  critique  provides  feedback  to  the  team  on 
how  well  it  performed  the  exercise.  This  feedback  can  only  be  effective 
if  two  conditions  are  met.  First,  the  instructors  must  systematically 
observe  and  record  team  performance  during  an  exercise.  Second,  they 
must  have  an  objective  basis  for  determining  the  adequacy  of  the  performance 
observed . 


The  systematic  observation  and  recording  of  team  performance 
requires  that  a measurement  system  be  available  for  the  exercises  conducted. 
This  serves  to  standardize  the  observing  of  performance  and  thus  is  designed 
to  assist  the  instructor  in  the  conduct  of  the  critique.  Thus,  measurement 
establishes  a basis  for  providing  the  team  knowledge  of  results  of  its  perform- 
ance, thereby  serving  a role  in  the  learning  process.  Measurement  also 
indicates  how  the  team  is  progressing  and  is  a basis  for  exercise  selection 
(training  strategy)  by  the  instructor  staff. 

A well- deve loped  measurement  system  (i.e.  , automated  scoring  and 
recording,  hard  copy  printouts  of  team  performance  for  critique  and  school 
record-keeping  functions)  is  desired,  since  this  could  enhance  training 
effectiveness  by  providing  an  array  of  objective  and  standardized  indicants 
of  performance.  The  advantage  of  this  for  developing  a strategy  for  training 
individual  teams  , for  refining  the  school  curriculum  and  for  maintaining 
quality  control  of  training  is  obvious. 

Note:  For  the  14A2  device,  the  decision  was  made  to 

provide  a minimal  scoring  and  recording  of  performance 
capability . 
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33A2  3«t  » f»»m  §*  pt &$** *>****% ■■  Tt*«*«  are  as  fallow# . 


# ■ff ■»-.<=  re**te#r+d«i<a*f..tMf  fewte  «»f  triiicat  evewf*  to  fee 

*«  c<w«*|»S**tt«4  *4*  ffee  e««fc**e  are  fe<or4cdl . Tfeese  are 

pr»*3d«»d  i4*  tle<MW  formal  for  e*<cH  r«ef«  *»e  <»wl  identify 
Hbm>  prwtf.edu  rbl  s**llvltle*  |<p<quipt»ie*>i1f.  operation  and 
(ommwnkMNMi)  *o  fee  performed  in  the  tomjwri* 

mewl*.,  Mao,  a *»(««* * leoal  display  IUPC5  replay  capability) 
of  wfee  loffil  mi»*ioo  and  1 toe  c ritiea!  «w»l*  ere  recorded  for 
ifee  critique.. 

a Quantitative  store#  of  lime  and  accuracy  in  developing  fire 
control  *oluiio«#  are  obtained*  Hardware  provision#  a-e  made 
for  Uw  following  display  of  information. 

* Fire  control  solution'  team*dct«rmined  target  course  and 
speed,  range/ bearing  which  can  fee  compared  with  actual 
target  range  and  bearing. 

- Digital  readouts  on  weapons?  how  close  the  fired  torpedo 
comes  to  the  target;  range/bearing,  depth,  course,  speed 
and  aspect  information  is  provided.  Water  entry  point 
(WEP)  in  relation  to  the  target  can  be  determined.  Also, 
control  and  attack  accuracy  (e.g. , MAD  accuracy)  is 
provided  on  the  ASAC  console. 

- Digital  readouts  of  time  of  attack  intervals:  elapsed  time 
from  contact  to  first  attack;  from  first  to  second  attack: 
and  from  second  to  third  attack  is  provided. 


# Instructor  judgments  arc  relied  upon  for  evaluating  the  more 
complex  segments  of  team  performance  involving  coordinated 
activities  among  team  members.  These  are  concerned  with 
the  team’s  ability  to  employ  itaeU  and  its  resources  in  the 
approach,  search,  and  contact  prosecution  phases  of  destroyer 
ASW  missions  (e.g.  , sensor  employment,  weapons  employment 
and  vectoring  or  positioning  of  units  under  control  by  the  team). 

In  addition,  team  coordination  is  judged  by  the  internal  and 
external  communications  discipline  and  by  the  quality  of  the 
communications  (content,  anticipation  of  needs  of  other  members, 
and  laten<  as  in  conveying  information). 


An  overall  team  score  or  grade  is  not  provided  for  an  exercise. 
The  assessment  purpose  is  to  determine  whether  or  not  the  team  achieved 
the  training  objectives  of  the  exercise.  Since  measurement  focuses  on  team 
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pr?  f format?*  **  «?val*M»li©n  Jot'  i*sg  ifo  tb*s  ever*.  sap  «.  F itiqtic . it  i«  tailored  to 
obtaining  those  performance  indie  aiions  that  tin  rpaliilitally  be  obtained 


The  training  standard*  that  apply  to  the  HA2  training  are  direc  tly 


related  to  the  three  part  measurement  system  just  discussed  (occurrence  of 
critical  events,  lime  and  accuracy  of  fire  control  solutions,  judgments  of 
pe  rformance) , 

4b„  10  Communications 


The  communications  capabilities  at  the  instructor  station  duplicate 


those  required  in  the  ASW  environment.  The  following  link*  have  common 
usage  among  all  subteam  compartments. 


. 21  MC,  27  MC,  29  MC,  intercom. 

• Sound  powered  telephones- - 1 JS  (ASW  control),  21 J3. 
22JS  (radar  announcing  circuit),  61  JS  (sonar  announcing 
circuit),  and  JA , JC,  8JP,  5JV. 

• RT  net#- -maximum  of  four  to  accommodate:  PRITAG, 

SAUTAC,  screen  common,  SAU  Cl,  ASW  common, 

ASW  air  control  net. 


Aleo,  provisions  arc  made  at  the  instructor  station  for  ASW  contact  alarm 
and  ASW  salvo  alarm  buzzer. 

4.6.11  Overall  Workplace  and  Layout  Requirements 


We  will  not  emphasize  the  human  engineering  design  and  layout  of 


the  14A2  instructor  station  complex  except  to  indicate  and  comment  on  basic 
layout  requirements.  An  ample  supply  of  human  engineering  design  stand- 
ards and  military  specifications  are  in  existence  and  are  applicable  to  this 
device  class.  Design  should  conform  to  these  specifications,  as  pertinent. 
Suffice  it  to  say  that  the  instructor  station  layout  should  be  accomplished 
in  modularized  components  consistent  with  primary  and  secondary  use  re- 
quirements and  vdth  established  human  engineering  design  practices. 


We  will,  however,  comment  on  a number  of  inadequacies  in  the 


instructor  station  layout  that  violate  good  design  practices  and  in  some 
instances  hamper  the  instructional  capability  . The  luxury  of  hindsight 
provides  us  with  an  assist  in  this  task. 
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4,6.  11,1  Phy » ic  .a  I L. » nl.  A loul  of  «>lcveti  consoles  a rrangcti  in  a 

•iiraight  lin<?  comprise  ih  Hsirntior  station  complex,  The?  arranucmcnl  and 
identification  of  the?  < ousel-  =•  ar**  *Hnwn  »ti  Figure*  54.  In  essence,  the*  physi- 
cal arrangemciu  is  eumoer-  nwe.  the?  consoles  are  aligned  side-by-side  over 
a considerable  area  and  it  la  difficult  for  one  instructor  to  use  the  controls  and 
displays  efficiently.  The  eleven  consoles  in  the  linear  arfay  cover  21  feet 
of  space,  each  console  is  46  inches  high  and  43  inches  deep.  There  are,  for 
example,  three  identical  consoles  (vehicle  control).  However,  in  the  present 
setup,  this  number  is  required  for  complex  multi-unit  tactical  problems. 


The  cumbersomcness  of  the  layoui  is  measured  in  a number  of 
design  inadequacies.  Representative  ones  are  outlined  below  to  provide  a 
flavor  of  the  types  of  features  that  burden  the  instructor.  (The  good  features 
of  design  are  tacitly  recognized  in  this  section.)  A number  of  control  and 
display  placement  features  are  identified  using  as  a referent  the  primary 
usage  in  relation  to  the  master  console. 

• The  fire  control  console  should  be  positioned  closer 
to  the  master  console  in  terms  of  usage;  the  fire 
control  and  the  vehicle  data  consoles  should  be 
together . 

. Weapon  presettings  should  be  on  the  master  console 
instead  of  on  the  Fire  Control  (computer  display) 
console . 

, The  seascape  console  is  positioned  too  remote  from 
the  master  console  and  cannot  be  seen  directly. 

, Controls  and  displays  for  handling  the  assist  ship 
and  flying  an  aircraft  are  too  far  apart  for  multi- 
unit exercises. 


The  event  symbol  for  the  projector  display  (environment 
console)  is  too  remote  from  the  master  console  since 
it  must  be  activated  for  every  piece  of  critical  informa- 
tion added  to  the  projection  display.  It  would  be  more 
advantageously  placed  on  the  master  console. 

Submarine  control  (vehicle  control  console)  should  be 
moved  so  that  the  operator  cannot  see  the  whole  tactical 
environment  as  it  develops  (which  is  an  advantage  to 
him  in  positioning  the  submarine). 


At  present,  weapon  readout  requires  multiple  button 
depressions  simultaneously.  A single  button  activation 
would  eliminate  this  cumbersome  feature. 
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4 t I I.»  Special  Design  Features.  The  instructor  co»ito!c  complex 

is  lee  filed  jti  ihe  problem  control  room  as  shown  below. 


Briefing 

Podium 


Projection  Display  (S'  x 8') 
(Situational  Display  of  the 
Mission) 


□ 


(Chairs  fill  the 
center  of  the  room) 


Instructor  Console  Complex 


A design  requirement  is  that  direct  visual  access  be  provided  to  the  projection 
display  throughout  the  running  of  a mission.  In  the  design  selected,  console 
heights  were  set  at  46  inches  from  the  floor  to  enable  instructors  to  see  the 
projection  display. 
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